2026-02-21T11:10:30.277 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T11:10:30.291 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T11:10:30.368 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62482 branch: wip-bharath21-testing-2026-02-20-1039 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '62482' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath21-testing-2026-02-20-1039 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 workunit: branch: wip-bharath21-testing-2026-02-20-1039 sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6481 sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath21-testing-2026-02-20-1039 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 targets: trial166.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAsHpT1xTW1qbjhsg+NkaQ8RIZGSSwRP552vLe5VmyFX6LtP4ebMeTmyXsTh/rv9Y2Cmmm+zd4Tuz6NJFE7x9w= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_10:47:12 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T11:10:30.368 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa; will attempt to use it 2026-02-21T11:10:30.368 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7273f3049146bdd7fa7b4a400114872a50228e9/qa/tasks 2026-02-21T11:10:30.368 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T11:10:30.370 INFO:teuthology.task.internal:Checking packages... 2026-02-21T11:10:30.370 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f7273f3049146bdd7fa7b4a400114872a50228e9' 2026-02-21T11:10:30.370 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T11:10:30.370 INFO:teuthology.packaging:ref: None 2026-02-21T11:10:30.370 INFO:teuthology.packaging:tag: None 2026-02-21T11:10:30.370 INFO:teuthology.packaging:branch: wip-bharath21-testing-2026-02-20-1039 2026-02-21T11:10:30.370 INFO:teuthology.packaging:sha1: f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:10:30.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath21-testing-2026-02-20-1039 2026-02-21T11:10:30.492 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:10:30.493 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T11:10:30.504 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T11:10:30.504 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T11:10:30.516 INFO:teuthology.task.internal:Saving configuration 2026-02-21T11:10:30.523 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T11:10:30.534 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T11:10:30.613 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial166.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62482', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 11:07:01.665954', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOHDO1z+DXJHNLXfcYKveMocrerJRKN+vbdsvPxGBpq2'} 2026-02-21T11:10:30.613 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T11:10:30.624 INFO:teuthology.task.internal:roles: ubuntu@trial166.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-21T11:10:30.624 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T11:10:30.709 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7b8b995f3880>, signals=[15]) 2026-02-21T11:10:30.709 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T11:10:30.720 INFO:teuthology.task.internal:Opening connections... 2026-02-21T11:10:30.720 DEBUG:teuthology.task.internal:connecting to ubuntu@trial166.front.sepia.ceph.com 2026-02-21T11:10:30.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T11:10:30.784 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T11:10:30.795 DEBUG:teuthology.orchestra.run.trial166:> uname -m 2026-02-21T11:10:30.861 INFO:teuthology.orchestra.run.trial166.stdout:x86_64 2026-02-21T11:10:30.862 DEBUG:teuthology.orchestra.run.trial166:> cat /etc/os-release 2026-02-21T11:10:30.905 INFO:teuthology.orchestra.run.trial166.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-21T11:10:30.905 INFO:teuthology.orchestra.run.trial166.stdout:NAME="Ubuntu" 2026-02-21T11:10:30.905 INFO:teuthology.orchestra.run.trial166.stdout:VERSION_ID="22.04" 2026-02-21T11:10:30.905 INFO:teuthology.orchestra.run.trial166.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-21T11:10:30.905 INFO:teuthology.orchestra.run.trial166.stdout:VERSION_CODENAME=jammy 2026-02-21T11:10:30.905 INFO:teuthology.orchestra.run.trial166.stdout:ID=ubuntu 2026-02-21T11:10:30.906 INFO:teuthology.orchestra.run.trial166.stdout:ID_LIKE=debian 2026-02-21T11:10:30.906 INFO:teuthology.orchestra.run.trial166.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-21T11:10:30.906 INFO:teuthology.orchestra.run.trial166.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-21T11:10:30.906 INFO:teuthology.orchestra.run.trial166.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-21T11:10:30.906 INFO:teuthology.orchestra.run.trial166.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-21T11:10:30.906 INFO:teuthology.orchestra.run.trial166.stdout:UBUNTU_CODENAME=jammy 2026-02-21T11:10:30.906 INFO:teuthology.lock.ops:Updating trial166.front.sepia.ceph.com on lock server 2026-02-21T11:10:31.001 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T11:10:31.013 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T11:10:31.025 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T11:10:31.026 DEBUG:teuthology.orchestra.run.trial166:> test '!' -e /home/ubuntu/cephtest 2026-02-21T11:10:31.029 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T11:10:31.040 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T11:10:31.041 DEBUG:teuthology.orchestra.run.trial166:> test -z $(ls -A /var/lib/ceph) 2026-02-21T11:10:31.074 INFO:teuthology.orchestra.run.trial166.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-21T11:10:31.075 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T11:10:31.176 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T11:10:31.197 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-21T11:10:31.197 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-21T11:10:31.197 DEBUG:teuthology.orchestra.run.trial166:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T11:10:31.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:10:31.201 DEBUG:teuthology.orchestra.run.trial166:> uname -r 2026-02-21T11:10:31.246 INFO:teuthology.orchestra.run.trial166.stdout:5.15.0-170-generic 2026-02-21T11:10:31.246 INFO:teuthology.task.kernel:Running kernel on trial166: 5.15.0-170-generic 2026-02-21T11:10:31.246 DEBUG:teuthology.orchestra.run.trial166:> sudo apt-get clean 2026-02-21T11:10:31.304 DEBUG:teuthology.orchestra.run.trial166:> sudo apt-get update 2026-02-21T11:10:31.828 INFO:teuthology.orchestra.run.trial166.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T11:10:31.912 INFO:teuthology.orchestra.run.trial166.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-21T11:10:32.261 INFO:teuthology.orchestra.run.trial166.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-21T11:10:32.410 INFO:teuthology.orchestra.run.trial166.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-21T11:10:32.452 INFO:teuthology.orchestra.run.trial166.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-21T11:10:32.513 INFO:teuthology.orchestra.run.trial166.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-21T11:10:32.526 INFO:teuthology.orchestra.run.trial166.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-21T11:10:32.536 INFO:teuthology.orchestra.run.trial166.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-21T11:10:32.552 INFO:teuthology.orchestra.run.trial166.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-21T11:10:32.564 INFO:teuthology.orchestra.run.trial166.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-21T11:10:32.574 INFO:teuthology.orchestra.run.trial166.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-21T11:10:32.611 INFO:teuthology.orchestra.run.trial166.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-21T11:10:32.626 INFO:teuthology.orchestra.run.trial166.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-21T11:10:32.652 INFO:teuthology.orchestra.run.trial166.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-21T11:10:32.664 INFO:teuthology.orchestra.run.trial166.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-21T11:10:32.674 INFO:teuthology.orchestra.run.trial166.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-21T11:10:32.689 INFO:teuthology.orchestra.run.trial166.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-21T11:10:32.699 INFO:teuthology.orchestra.run.trial166.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-21T11:10:32.786 INFO:teuthology.orchestra.run.trial166.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-21T11:10:36.486 INFO:teuthology.orchestra.run.trial166.stdout:Fetched 22.6 MB in 2s (9068 kB/s) 2026-02-21T11:10:37.020 INFO:teuthology.orchestra.run.trial166.stdout:Reading package lists... 2026-02-21T11:10:37.034 DEBUG:teuthology.orchestra.run.trial166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-21T11:10:37.066 INFO:teuthology.orchestra.run.trial166.stdout:Reading package lists... 2026-02-21T11:10:37.174 INFO:teuthology.orchestra.run.trial166.stdout:Building dependency tree... 2026-02-21T11:10:37.174 INFO:teuthology.orchestra.run.trial166.stdout:Reading state information... 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout:linux-image-generic set to manually installed. 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout: sg3-utils-udev 2026-02-21T11:10:37.261 INFO:teuthology.orchestra.run.trial166.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T11:10:37.281 INFO:teuthology.orchestra.run.trial166.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T11:10:37.282 DEBUG:teuthology.orchestra.run.trial166:> dpkg -s linux-image-generic 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Package: linux-image-generic 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Status: install ok installed 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Priority: optional 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Section: kernel 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Installed-Size: 21 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Maintainer: Ubuntu Kernel Team 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Architecture: amd64 2026-02-21T11:10:37.335 INFO:teuthology.orchestra.run.trial166.stdout:Source: linux-meta 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout:Version: 5.15.0.170.159 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout:Recommends: thermald 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout:Description: Generic Linux kernel image 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout: This package will always depend on the latest generic kernel image 2026-02-21T11:10:37.336 INFO:teuthology.orchestra.run.trial166.stdout: available. 2026-02-21T11:10:37.338 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T11:10:37.338 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T11:10:37.338 DEBUG:teuthology.orchestra.run.trial166:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T11:10:37.386 INFO:teuthology.orchestra.run.trial166.stdout:ttyS1 2026-02-21T11:10:37.421 DEBUG:teuthology.parallel:result is None 2026-02-21T11:10:37.421 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T11:10:37.435 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T11:10:37.435 DEBUG:teuthology.orchestra.run.trial166:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T11:10:37.465 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T11:10:37.476 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T11:10:37.489 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T11:10:37.489 DEBUG:teuthology.orchestra.run.trial166:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T11:10:37.511 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T11:10:37.521 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T11:10:37.521 DEBUG:teuthology.orchestra.run.trial166:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T11:10:37.558 INFO:teuthology.orchestra.run.trial166.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:10:37.565 INFO:teuthology.orchestra.run.trial166.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:10:37.566 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T11:10:37.576 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T11:10:37.576 DEBUG:teuthology.orchestra.run.trial166:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T11:10:37.618 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T11:10:37.637 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T11:10:37.637 DEBUG:teuthology.orchestra.run.trial166:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T11:10:37.666 DEBUG:teuthology.orchestra.run.trial166:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T11:10:37.709 DEBUG:teuthology.orchestra.run.trial166:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T11:10:37.753 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:10:37.753 DEBUG:teuthology.orchestra.run.trial166:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T11:10:37.802 DEBUG:teuthology.orchestra.run.trial166:> sudo service rsyslog restart 2026-02-21T11:10:37.860 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T11:10:37.873 INFO:teuthology.task.internal:Starting timer... 2026-02-21T11:10:37.873 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T11:10:37.894 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T11:10:37.915 DEBUG:teuthology.task.selinux:Excluding trial166: OS 'ubuntu' does not support SELinux 2026-02-21T11:10:37.916 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T11:10:37.916 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-21T11:10:37.916 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T11:10:37.916 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T11:10:37.931 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T11:10:37.943 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T11:10:37.944 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial166.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T11:12:58.012 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial166.front.sepia.ceph.com')] 2026-02-21T11:12:58.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial166.front.sepia.ceph.com' 2026-02-21T11:12:58.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial166.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T11:12:58.123 DEBUG:teuthology.orchestra.run.trial166:> true 2026-02-21T11:12:58.374 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial166.front.sepia.ceph.com' 2026-02-21T11:12:58.375 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T11:12:58.392 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T11:12:58.392 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T11:12:58.393 DEBUG:teuthology.orchestra.run.trial166:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T11:12:58.437 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-21T11:12:58.437 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Command line: ntpd -gq 2026-02-21T11:12:58.437 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: ---------------------------------------------------- 2026-02-21T11:12:58.437 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: ntp-4 is maintained by Network Time Foundation, 2026-02-21T11:12:58.438 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-21T11:12:58.438 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: corporation. Support and training for ntp-4 are 2026-02-21T11:12:58.438 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: available at https://www.nwtime.org/support 2026-02-21T11:12:58.438 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: ---------------------------------------------------- 2026-02-21T11:12:58.438 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: proto: precision = 0.020 usec (-25) 2026-02-21T11:12:58.438 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: basedate set to 2022-02-04 2026-02-21T11:12:58.439 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: gps base set to 2022-02-06 (week 2196) 2026-02-21T11:12:58.439 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T11:12:58.439 INFO:teuthology.orchestra.run.trial166.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T11:12:58.440 INFO:teuthology.orchestra.run.trial166.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-21T11:12:58.442 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: restrict ::: KOD does nothing without LIMITED. 2026-02-21T11:12:58.442 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listen and drop on 0 v6wildcard [::]:123 2026-02-21T11:12:58.442 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-21T11:12:58.442 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listen normally on 2 lo 127.0.0.1:123 2026-02-21T11:12:58.442 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listen normally on 3 enp1s0f0 10.20.193.166:123 2026-02-21T11:12:58.443 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listen normally on 4 lo [::1]:123 2026-02-21T11:12:58.443 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:631e%2]:123 2026-02-21T11:12:58.443 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:12:58 ntpd[9411]: Listening on routing socket on fd #22 for interface updates 2026-02-21T11:12:59.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:12:59.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:12:59.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:12:59.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:12:59.469 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:12:59.469 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:12:59.469 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:12:59.469 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:12:59.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:12:59.501 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:12:59.501 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:12:59.501 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:12:59 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:01.435 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:01.435 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:01.435 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:01.435 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:01.435 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:01.435 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:01.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:01.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:01.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:01.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:01.475 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:01.475 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:01.475 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:01.476 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:01.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:01.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:01.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:01.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:01 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:03.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.437 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.437 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.437 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.437 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.437 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.455 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.455 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.455 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:03.455 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:03.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:03.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:03.475 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.475 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.476 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:03.476 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:03.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:03.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:03.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:03.500 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:03 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:05.436 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:05.454 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:05.454 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:05.454 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:05.454 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stdout:21 Feb 11:13:05 ntpd[9411]: ntpd: time set +0.039338 s 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stdout:ntpd: time set +0.039338s 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-21T11:13:05.468 INFO:teuthology.orchestra.run.trial166.stderr:21 Feb 11:13:05 ntpd[9411]: can't open /var/log/ntpstats/loopstats.20260221: Permission denied 2026-02-21T11:13:05.552 INFO:teuthology.orchestra.run.trial166.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T11:13:05.552 INFO:teuthology.orchestra.run.trial166.stdout:============================================================================== 2026-02-21T11:13:05.552 INFO:teuthology.orchestra.run.trial166.stdout: time-he.den.cod .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T11:13:05.552 INFO:teuthology.orchestra.run.trial166.stdout: 44.190.5.123 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T11:13:05.552 INFO:teuthology.orchestra.run.trial166.stdout: eterna.binary.n .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T11:13:05.553 INFO:teuthology.orchestra.run.trial166.stdout: chi3.us.ntp.li .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T11:13:05.553 INFO:teuthology.run_tasks:Running task install... 2026-02-21T11:13:05.565 DEBUG:teuthology.task.install:project ceph 2026-02-21T11:13:05.566 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'}} 2026-02-21T11:13:05.566 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f7273f3049146bdd7fa7b4a400114872a50228e9'} 2026-02-21T11:13:05.566 INFO:teuthology.task.install:Using flavor: default 2026-02-21T11:13:05.571 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T11:13:05.571 INFO:teuthology.task.install:extra packages: [] 2026-02-21T11:13:05.571 DEBUG:teuthology.orchestra.run.trial166:> sudo apt-key list | grep Ceph 2026-02-21T11:13:05.640 INFO:teuthology.orchestra.run.trial166.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-21T11:13:05.668 INFO:teuthology.orchestra.run.trial166.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-21T11:13:05.668 INFO:teuthology.orchestra.run.trial166.stdout:uid [ unknown] Ceph.com (release key) 2026-02-21T11:13:05.670 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-21T11:13:05.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:13:05.843 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default/ 2026-02-21T11:13:05.843 INFO:teuthology.task.install.deb:Package version is 20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:13:05.988 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:13:05.988 DEBUG:teuthology.orchestra.run.trial166:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-21T11:13:05.999 DEBUG:teuthology.orchestra.run.trial166:> sudo apt-get update 2026-02-21T11:13:06.158 INFO:teuthology.orchestra.run.trial166.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy InRelease 2026-02-21T11:13:06.169 INFO:teuthology.orchestra.run.trial166.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-21T11:13:06.182 INFO:teuthology.orchestra.run.trial166.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-21T11:13:06.193 INFO:teuthology.orchestra.run.trial166.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-21T11:13:06.519 INFO:teuthology.orchestra.run.trial166.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T11:13:06.605 INFO:teuthology.orchestra.run.trial166.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-21T11:13:06.697 INFO:teuthology.orchestra.run.trial166.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-21T11:13:06.789 INFO:teuthology.orchestra.run.trial166.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-21T11:13:06.928 INFO:teuthology.orchestra.run.trial166.stdout:Fetched 26.9 kB in 1s (35.0 kB/s) 2026-02-21T11:13:07.445 INFO:teuthology.orchestra.run.trial166.stdout:Reading package lists... 2026-02-21T11:13:07.457 DEBUG:teuthology.orchestra.run.trial166:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5397-gf7273f30-1jammy cephadm=20.3.0-5397-gf7273f30-1jammy ceph-mds=20.3.0-5397-gf7273f30-1jammy ceph-mgr=20.3.0-5397-gf7273f30-1jammy ceph-osd=20.3.0-5397-gf7273f30-1jammy ceph-osd-classic=20.3.0-5397-gf7273f30-1jammy ceph-common=20.3.0-5397-gf7273f30-1jammy ceph-fuse=20.3.0-5397-gf7273f30-1jammy ceph-test=20.3.0-5397-gf7273f30-1jammy ceph-volume=20.3.0-5397-gf7273f30-1jammy radosgw=20.3.0-5397-gf7273f30-1jammy python3-rados=20.3.0-5397-gf7273f30-1jammy python3-rgw=20.3.0-5397-gf7273f30-1jammy python3-cephfs=20.3.0-5397-gf7273f30-1jammy python3-rbd=20.3.0-5397-gf7273f30-1jammy libcephfs2=20.3.0-5397-gf7273f30-1jammy libcephfs-dev=20.3.0-5397-gf7273f30-1jammy librados2=20.3.0-5397-gf7273f30-1jammy librbd1=20.3.0-5397-gf7273f30-1jammy rbd-fuse=20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:13:07.491 INFO:teuthology.orchestra.run.trial166.stdout:Reading package lists... 2026-02-21T11:13:07.597 INFO:teuthology.orchestra.run.trial166.stdout:Building dependency tree... 2026-02-21T11:13:07.597 INFO:teuthology.orchestra.run.trial166.stdout:Reading state information... 2026-02-21T11:13:07.687 INFO:teuthology.orchestra.run.trial166.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout: sg3-utils-udev 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout:The following additional packages will be installed: 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-21T11:13:07.688 INFO:teuthology.orchestra.run.trial166.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-21T11:13:07.689 INFO:teuthology.orchestra.run.trial166.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-21T11:13:07.690 INFO:teuthology.orchestra.run.trial166.stdout:Suggested packages: 2026-02-21T11:13:07.690 INFO:teuthology.orchestra.run.trial166.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-21T11:13:07.690 INFO:teuthology.orchestra.run.trial166.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-21T11:13:07.690 INFO:teuthology.orchestra.run.trial166.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-21T11:13:07.690 INFO:teuthology.orchestra.run.trial166.stdout:Recommended packages: 2026-02-21T11:13:07.690 INFO:teuthology.orchestra.run.trial166.stdout: btrfs-tools 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout:The following NEW packages will be installed: 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T11:13:07.722 INFO:teuthology.orchestra.run.trial166.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-21T11:13:07.723 INFO:teuthology.orchestra.run.trial166.stdout: socat xmlstarlet 2026-02-21T11:13:07.801 INFO:teuthology.orchestra.run.trial166.stdout:0 upgraded, 87 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T11:13:07.801 INFO:teuthology.orchestra.run.trial166.stdout:Need to get 699 MB of archives. 2026-02-21T11:13:07.801 INFO:teuthology.orchestra.run.trial166.stdout:After this operation, 2661 MB of additional disk space will be used. 2026-02-21T11:13:07.801 INFO:teuthology.orchestra.run.trial166.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5397-gf7273f30-1jammy [3794 kB] 2026-02-21T11:13:07.907 INFO:teuthology.orchestra.run.trial166.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5397-gf7273f30-1jammy [2910 kB] 2026-02-21T11:13:07.947 INFO:teuthology.orchestra.run.trial166.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5397-gf7273f30-1jammy [882 kB] 2026-02-21T11:13:07.960 INFO:teuthology.orchestra.run.trial166.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5397-gf7273f30-1jammy [359 kB] 2026-02-21T11:13:07.966 INFO:teuthology.orchestra.run.trial166.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5397-gf7273f30-1jammy [32.8 kB] 2026-02-21T11:13:07.967 INFO:teuthology.orchestra.run.trial166.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5397-gf7273f30-1jammy [225 kB] 2026-02-21T11:13:07.971 INFO:teuthology.orchestra.run.trial166.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5397-gf7273f30-1jammy [88.4 kB] 2026-02-21T11:13:07.972 INFO:teuthology.orchestra.run.trial166.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5397-gf7273f30-1jammy [338 kB] 2026-02-21T11:13:07.978 INFO:teuthology.orchestra.run.trial166.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5397-gf7273f30-1jammy [9089 kB] 2026-02-21T11:13:08.074 INFO:teuthology.orchestra.run.trial166.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-21T11:13:08.128 INFO:teuthology.orchestra.run.trial166.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5397-gf7273f30-1jammy [112 kB] 2026-02-21T11:13:08.128 INFO:teuthology.orchestra.run.trial166.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5397-gf7273f30-1jammy [261 kB] 2026-02-21T11:13:08.132 INFO:teuthology.orchestra.run.trial166.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5397-gf7273f30-1jammy [31.5 MB] 2026-02-21T11:13:08.412 INFO:teuthology.orchestra.run.trial166.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-21T11:13:08.422 INFO:teuthology.orchestra.run.trial166.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-21T11:13:08.491 INFO:teuthology.orchestra.run.trial166.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-21T11:13:08.654 INFO:teuthology.orchestra.run.trial166.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5397-gf7273f30-1jammy [4984 kB] 2026-02-21T11:13:08.693 INFO:teuthology.orchestra.run.trial166.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-21T11:13:08.703 INFO:teuthology.orchestra.run.trial166.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-21T11:13:08.730 INFO:teuthology.orchestra.run.trial166.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-21T11:13:08.731 INFO:teuthology.orchestra.run.trial166.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5397-gf7273f30-1jammy [294 kB] 2026-02-21T11:13:08.735 INFO:teuthology.orchestra.run.trial166.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5397-gf7273f30-1jammy [127 kB] 2026-02-21T11:13:08.737 INFO:teuthology.orchestra.run.trial166.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5397-gf7273f30-1jammy [934 kB] 2026-02-21T11:13:08.738 INFO:teuthology.orchestra.run.trial166.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-21T11:13:08.741 INFO:teuthology.orchestra.run.trial166.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-21T11:13:08.741 INFO:teuthology.orchestra.run.trial166.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-21T11:13:08.742 INFO:teuthology.orchestra.run.trial166.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-21T11:13:08.753 INFO:teuthology.orchestra.run.trial166.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5397-gf7273f30-1jammy [7538 kB] 2026-02-21T11:13:08.759 INFO:teuthology.orchestra.run.trial166.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-21T11:13:08.760 INFO:teuthology.orchestra.run.trial166.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-21T11:13:08.766 INFO:teuthology.orchestra.run.trial166.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-21T11:13:08.832 INFO:teuthology.orchestra.run.trial166.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-21T11:13:08.832 INFO:teuthology.orchestra.run.trial166.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-21T11:13:08.832 INFO:teuthology.orchestra.run.trial166.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-21T11:13:08.833 INFO:teuthology.orchestra.run.trial166.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-21T11:13:08.833 INFO:teuthology.orchestra.run.trial166.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-21T11:13:08.833 INFO:teuthology.orchestra.run.trial166.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-21T11:13:08.833 INFO:teuthology.orchestra.run.trial166.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-21T11:13:08.838 INFO:teuthology.orchestra.run.trial166.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-21T11:13:08.838 INFO:teuthology.orchestra.run.trial166.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-21T11:13:08.877 INFO:teuthology.orchestra.run.trial166.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5397-gf7273f30-1jammy [468 MB] 2026-02-21T11:13:08.907 INFO:teuthology.orchestra.run.trial166.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-21T11:13:08.909 INFO:teuthology.orchestra.run.trial166.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-21T11:13:08.910 INFO:teuthology.orchestra.run.trial166.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-21T11:13:08.910 INFO:teuthology.orchestra.run.trial166.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-21T11:13:08.936 INFO:teuthology.orchestra.run.trial166.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-21T11:13:08.937 INFO:teuthology.orchestra.run.trial166.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-21T11:13:08.938 INFO:teuthology.orchestra.run.trial166.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-21T11:13:08.996 INFO:teuthology.orchestra.run.trial166.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-21T11:13:08.996 INFO:teuthology.orchestra.run.trial166.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-21T11:13:08.997 INFO:teuthology.orchestra.run.trial166.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-21T11:13:08.997 INFO:teuthology.orchestra.run.trial166.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-21T11:13:08.997 INFO:teuthology.orchestra.run.trial166.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-21T11:13:08.998 INFO:teuthology.orchestra.run.trial166.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-21T11:13:09.056 INFO:teuthology.orchestra.run.trial166.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-21T11:13:09.057 INFO:teuthology.orchestra.run.trial166.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-21T11:13:09.058 INFO:teuthology.orchestra.run.trial166.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-21T11:13:09.128 INFO:teuthology.orchestra.run.trial166.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-21T11:13:09.131 INFO:teuthology.orchestra.run.trial166.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-21T11:13:09.133 INFO:teuthology.orchestra.run.trial166.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-21T11:13:09.137 INFO:teuthology.orchestra.run.trial166.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-21T11:13:09.140 INFO:teuthology.orchestra.run.trial166.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-21T11:13:09.140 INFO:teuthology.orchestra.run.trial166.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-21T11:13:09.141 INFO:teuthology.orchestra.run.trial166.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-21T11:13:09.201 INFO:teuthology.orchestra.run.trial166.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-21T11:13:09.201 INFO:teuthology.orchestra.run.trial166.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-21T11:13:09.207 INFO:teuthology.orchestra.run.trial166.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-21T11:13:09.273 INFO:teuthology.orchestra.run.trial166.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-21T11:13:09.275 INFO:teuthology.orchestra.run.trial166.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-21T11:13:09.276 INFO:teuthology.orchestra.run.trial166.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-21T11:13:09.276 INFO:teuthology.orchestra.run.trial166.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-21T11:13:16.773 INFO:teuthology.orchestra.run.trial166.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5397-gf7273f30-1jammy [180 kB] 2026-02-21T11:13:16.775 INFO:teuthology.orchestra.run.trial166.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5397-gf7273f30-1jammy [14.0 kB] 2026-02-21T11:13:16.775 INFO:teuthology.orchestra.run.trial166.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5397-gf7273f30-1jammy [1030 kB] 2026-02-21T11:13:16.780 INFO:teuthology.orchestra.run.trial166.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5397-gf7273f30-1jammy [2437 kB] 2026-02-21T11:13:16.792 INFO:teuthology.orchestra.run.trial166.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5397-gf7273f30-1jammy [1068 kB] 2026-02-21T11:13:16.796 INFO:teuthology.orchestra.run.trial166.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5397-gf7273f30-1jammy [190 kB] 2026-02-21T11:13:16.797 INFO:teuthology.orchestra.run.trial166.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5397-gf7273f30-1jammy [54.8 MB] 2026-02-21T11:13:17.694 INFO:teuthology.orchestra.run.trial166.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5397-gf7273f30-1jammy [8626 kB] 2026-02-21T11:13:17.828 INFO:teuthology.orchestra.run.trial166.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5397-gf7273f30-1jammy [14.2 kB] 2026-02-21T11:13:17.828 INFO:teuthology.orchestra.run.trial166.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5397-gf7273f30-1jammy [65.4 MB] 2026-02-21T11:13:18.910 INFO:teuthology.orchestra.run.trial166.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5397-gf7273f30-1jammy [137 kB] 2026-02-21T11:13:18.911 INFO:teuthology.orchestra.run.trial166.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5397-gf7273f30-1jammy [45.8 kB] 2026-02-21T11:13:18.911 INFO:teuthology.orchestra.run.trial166.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5397-gf7273f30-1jammy [32.4 kB] 2026-02-21T11:13:18.912 INFO:teuthology.orchestra.run.trial166.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5397-gf7273f30-1jammy [44.2 kB] 2026-02-21T11:13:18.912 INFO:teuthology.orchestra.run.trial166.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5397-gf7273f30-1jammy [18.3 MB] 2026-02-21T11:13:19.203 INFO:teuthology.orchestra.run.trial166.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath21-testing-2026-02-20-1039/f7273f3049146bdd7fa7b4a400114872a50228e9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5397-gf7273f30-1jammy [93.7 kB] 2026-02-21T11:13:19.479 INFO:teuthology.orchestra.run.trial166.stdout:Fetched 699 MB in 11s (60.9 MB/s) 2026-02-21T11:13:19.495 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-21T11:13:19.650 INFO:teuthology.orchestra.run.trial166.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-21T11:13:19.652 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-21T11:13:19.654 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T11:13:19.672 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-21T11:13:19.685 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-21T11:13:19.685 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T11:13:19.702 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-21T11:13:19.715 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-21T11:13:19.715 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T11:13:19.732 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-21T11:13:19.740 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T11:13:19.745 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:13:19.775 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-21T11:13:19.783 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T11:13:19.783 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:13:19.799 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-21T11:13:19.808 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T11:13:19.808 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:13:19.830 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-21T11:13:19.838 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-21T11:13:19.838 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T11:13:19.854 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package librados2. 2026-02-21T11:13:19.862 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../07-librados2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:19.862 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking librados2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:19.903 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libnbd0. 2026-02-21T11:13:19.912 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-21T11:13:19.912 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-21T11:13:19.927 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package librbd1. 2026-02-21T11:13:19.935 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../09-librbd1_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:19.935 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking librbd1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:19.971 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libcephfs2. 2026-02-21T11:13:19.979 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:19.980 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libcephfs2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:19.999 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-rados. 2026-02-21T11:13:20.007 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.007 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-rados (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.025 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-21T11:13:20.033 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:20.033 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-ceph-argparse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.047 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-cephfs. 2026-02-21T11:13:20.055 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.056 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-cephfs (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.071 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-21T11:13:20.079 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:20.079 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.097 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-21T11:13:20.105 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-21T11:13:20.105 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T11:13:20.120 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-prettytable. 2026-02-21T11:13:20.128 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-21T11:13:20.128 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-21T11:13:20.143 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-rbd. 2026-02-21T11:13:20.151 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.152 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-rbd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.169 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-21T11:13:20.176 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-21T11:13:20.177 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T11:13:20.194 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package librgw2. 2026-02-21T11:13:20.202 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../19-librgw2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.202 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking librgw2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.299 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-rgw. 2026-02-21T11:13:20.307 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.308 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-rgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.323 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-21T11:13:20.330 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-21T11:13:20.331 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T11:13:20.345 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libradosstriper1. 2026-02-21T11:13:20.353 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.354 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libradosstriper1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.370 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-common. 2026-02-21T11:13:20.378 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.378 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.633 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-base. 2026-02-21T11:13:20.641 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.646 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-base (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.706 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-21T11:13:20.714 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-21T11:13:20.714 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-21T11:13:20.728 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-cheroot. 2026-02-21T11:13:20.736 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-21T11:13:20.736 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T11:13:20.752 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-21T11:13:20.760 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-21T11:13:20.761 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-21T11:13:20.775 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-21T11:13:20.784 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-21T11:13:20.784 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-21T11:13:20.798 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-21T11:13:20.806 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-21T11:13:20.806 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-21T11:13:20.820 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-tempora. 2026-02-21T11:13:20.829 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-21T11:13:20.829 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-21T11:13:20.844 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-portend. 2026-02-21T11:13:20.852 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-21T11:13:20.852 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-21T11:13:20.867 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-21T11:13:20.875 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-21T11:13:20.876 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-21T11:13:20.890 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-21T11:13:20.899 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-21T11:13:20.899 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-21T11:13:20.922 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-natsort. 2026-02-21T11:13:20.930 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-21T11:13:20.930 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-21T11:13:20.946 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-21T11:13:20.954 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:20.954 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:20.985 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-21T11:13:20.992 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:20.993 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:21.007 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mgr. 2026-02-21T11:13:21.016 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:21.016 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mgr (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:21.037 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mon. 2026-02-21T11:13:21.045 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:21.045 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:21.110 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-21T11:13:21.119 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-21T11:13:21.119 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T11:13:21.135 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-21T11:13:21.143 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:21.143 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-osd-classic (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.101 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-osd. 2026-02-21T11:13:25.113 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:25.113 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-osd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.133 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph. 2026-02-21T11:13:25.145 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../42-ceph_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:25.146 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.161 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-fuse. 2026-02-21T11:13:25.173 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:25.174 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.197 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mds. 2026-02-21T11:13:25.207 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:25.208 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mds (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.240 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package cephadm. 2026-02-21T11:13:25.248 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../45-cephadm_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:25.248 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.264 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-21T11:13:25.272 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T11:13:25.272 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T11:13:25.292 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-21T11:13:25.300 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:25.301 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:25.321 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-21T11:13:25.329 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-21T11:13:25.329 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-21T11:13:25.344 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-routes. 2026-02-21T11:13:25.353 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-21T11:13:25.353 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T11:13:25.372 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-21T11:13:25.380 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:25.380 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:26.064 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-21T11:13:26.074 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-21T11:13:26.074 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T11:13:26.112 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-joblib. 2026-02-21T11:13:26.120 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-21T11:13:26.121 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T11:13:26.147 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-21T11:13:26.156 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-21T11:13:26.156 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-21T11:13:26.171 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-sklearn. 2026-02-21T11:13:26.179 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-21T11:13:26.180 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T11:13:26.261 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-21T11:13:26.270 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:26.271 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:26.428 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-cachetools. 2026-02-21T11:13:26.436 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-21T11:13:26.437 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-21T11:13:26.451 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-rsa. 2026-02-21T11:13:26.460 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-21T11:13:26.460 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-21T11:13:26.477 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-google-auth. 2026-02-21T11:13:26.486 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-21T11:13:26.486 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-21T11:13:26.504 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-21T11:13:26.512 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-21T11:13:26.512 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T11:13:26.528 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-websocket. 2026-02-21T11:13:26.536 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-21T11:13:26.536 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-21T11:13:26.553 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-21T11:13:26.562 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-21T11:13:26.562 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T11:13:26.658 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-21T11:13:26.667 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:26.667 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:26.680 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-21T11:13:26.689 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-21T11:13:26.689 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T11:13:26.704 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-21T11:13:26.712 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T11:13:26.713 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T11:13:26.727 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package jq. 2026-02-21T11:13:26.736 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T11:13:26.736 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-21T11:13:26.750 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package socat. 2026-02-21T11:13:26.758 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-21T11:13:26.759 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-21T11:13:26.777 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package xmlstarlet. 2026-02-21T11:13:26.785 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-21T11:13:26.785 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-21T11:13:26.820 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-test. 2026-02-21T11:13:26.828 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:26.829 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-test (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:27.436 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package ceph-volume. 2026-02-21T11:13:27.444 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5397-gf7273f30-1jammy_all.deb ... 2026-02-21T11:13:27.445 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking ceph-volume (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:27.467 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-21T11:13:27.476 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:27.476 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libcephfs-daemon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:27.490 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-21T11:13:27.499 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:27.499 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libcephfs-proxy2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:27.513 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-21T11:13:27.522 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:27.522 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking libcephfs-dev (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:27.538 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package nvme-cli. 2026-02-21T11:13:27.546 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-21T11:13:27.547 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T11:13:27.577 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-21T11:13:27.585 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T11:13:27.586 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T11:13:27.614 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-21T11:13:27.623 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-21T11:13:27.624 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-21T11:13:27.637 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-pluggy. 2026-02-21T11:13:27.645 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-21T11:13:27.646 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-21T11:13:27.661 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-psutil. 2026-02-21T11:13:27.670 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-21T11:13:27.670 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-21T11:13:27.689 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-py. 2026-02-21T11:13:27.697 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-21T11:13:27.697 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-21T11:13:27.717 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-pygments. 2026-02-21T11:13:27.726 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-21T11:13:27.726 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T11:13:27.768 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-toml. 2026-02-21T11:13:27.777 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-21T11:13:27.777 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-21T11:13:27.791 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-pytest. 2026-02-21T11:13:27.800 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-21T11:13:27.800 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T11:13:27.821 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-simplejson. 2026-02-21T11:13:27.830 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-21T11:13:27.830 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-21T11:13:27.855 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package python3-webob. 2026-02-21T11:13:27.863 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-21T11:13:27.863 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T11:13:27.879 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-21T11:13:27.888 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-21T11:13:27.888 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-21T11:13:27.958 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package radosgw. 2026-02-21T11:13:27.967 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../85-radosgw_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:27.967 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking radosgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:28.131 INFO:teuthology.orchestra.run.trial166.stdout:Selecting previously unselected package rbd-fuse. 2026-02-21T11:13:28.139 INFO:teuthology.orchestra.run.trial166.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5397-gf7273f30-1jammy_amd64.deb ... 2026-02-21T11:13:28.139 INFO:teuthology.orchestra.run.trial166.stdout:Unpacking rbd-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:28.167 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-21T11:13:28.233 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T11:13:28.234 INFO:teuthology.orchestra.run.trial166.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T11:13:28.305 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T11:13:28.586 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-21T11:13:28.993 INFO:teuthology.orchestra.run.trial166.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-21T11:13:29.003 INFO:teuthology.orchestra.run.trial166.stdout:Setting up cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:29.047 INFO:teuthology.orchestra.run.trial166.stdout:Adding system user cephadm....done 2026-02-21T11:13:29.052 INFO:teuthology.orchestra.run.trial166.stdout:usermod: no changes 2026-02-21T11:13:29.057 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-21T11:13:29.129 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T11:13:29.130 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-21T11:13:29.204 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-21T11:13:29.278 INFO:teuthology.orchestra.run.trial166.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T11:13:29.279 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-21T11:13:29.373 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T11:13:29.497 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-21T11:13:29.570 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-21T11:13:29.645 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-ceph-argparse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:29.713 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T11:13:29.714 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-21T11:13:29.715 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T11:13:29.715 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T11:13:29.716 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-21T11:13:29.836 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-21T11:13:29.912 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libcephfs-proxy2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:29.913 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T11:13:29.993 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-21T11:13:30.077 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T11:13:30.318 INFO:teuthology.orchestra.run.trial166.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-21T11:13:30.318 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T11:13:30.412 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T11:13:30.535 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T11:13:30.615 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-21T11:13:30.688 INFO:teuthology.orchestra.run.trial166.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-21T11:13:30.689 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:30.777 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T11:13:31.270 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:13:31.275 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-21T11:13:31.351 INFO:teuthology.orchestra.run.trial166.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T11:13:31.352 INFO:teuthology.orchestra.run.trial166.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-21T11:13:31.353 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-21T11:13:31.429 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-21T11:13:31.499 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:13:31.500 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-21T11:13:31.580 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-21T11:13:31.653 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-21T11:13:31.733 INFO:teuthology.orchestra.run.trial166.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T11:13:31.734 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-21T11:13:31.818 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T11:13:31.819 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T11:13:31.897 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T11:13:31.977 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-21T11:13:32.057 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T11:13:32.058 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T11:13:32.192 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-21T11:13:32.263 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T11:13:32.264 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-21T11:13:32.347 INFO:teuthology.orchestra.run.trial166.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-21T11:13:32.347 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-21T11:13:32.479 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T11:13:32.480 INFO:teuthology.orchestra.run.trial166.stdout:Setting up librados2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:32.481 INFO:teuthology.orchestra.run.trial166.stdout:Setting up librgw2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:32.482 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:32.483 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T11:13:33.002 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libcephfs2 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.003 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libradosstriper1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.004 INFO:teuthology.orchestra.run.trial166.stdout:Setting up librbd1 (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.005 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mgr-modules-core (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.006 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.079 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T11:13:33.080 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T11:13:33.489 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libcephfs-dev (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.489 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-rados (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.490 INFO:teuthology.orchestra.run.trial166.stdout:Setting up libcephfs-daemon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.491 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-rbd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.492 INFO:teuthology.orchestra.run.trial166.stdout:Setting up rbd-fuse (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.493 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-rgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.494 INFO:teuthology.orchestra.run.trial166.stdout:Setting up python3-cephfs (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.494 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-common (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:33.524 INFO:teuthology.orchestra.run.trial166.stdout:Adding group ceph....done 2026-02-21T11:13:33.559 INFO:teuthology.orchestra.run.trial166.stdout:Adding system user ceph....done 2026-02-21T11:13:33.569 INFO:teuthology.orchestra.run.trial166.stdout:Setting system user ceph properties....done 2026-02-21T11:13:33.576 INFO:teuthology.orchestra.run.trial166.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-21T11:13:33.665 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-21T11:13:33.950 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-21T11:13:34.343 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-test (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:34.344 INFO:teuthology.orchestra.run.trial166.stdout:Setting up radosgw (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:34.609 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T11:13:34.609 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T11:13:34.989 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-base (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:35.108 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-21T11:13:35.482 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mds (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:35.541 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T11:13:35.541 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T11:13:35.901 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mgr (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:35.974 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T11:13:35.974 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T11:13:36.348 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.349 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-osd-classic (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.366 INFO:teuthology.orchestra.run.trial166.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-21T11:13:36.377 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.387 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mon (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.457 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T11:13:36.457 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T11:13:36.843 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mgr-cephadm (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.854 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-mgr-dashboard (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.865 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-osd (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:36.948 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T11:13:36.949 INFO:teuthology.orchestra.run.trial166.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T11:13:37.332 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:37.333 INFO:teuthology.orchestra.run.trial166.stdout:Setting up ceph-volume (20.3.0-5397-gf7273f30-1jammy) ... 2026-02-21T11:13:37.441 INFO:teuthology.orchestra.run.trial166.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-21T11:13:37.523 INFO:teuthology.orchestra.run.trial166.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-21T11:13:38.194 INFO:teuthology.orchestra.run.trial166.stdout: 2026-02-21T11:13:38.194 INFO:teuthology.orchestra.run.trial166.stdout:Running kernel seems to be up-to-date. 2026-02-21T11:13:38.194 INFO:teuthology.orchestra.run.trial166.stdout: 2026-02-21T11:13:38.194 INFO:teuthology.orchestra.run.trial166.stdout:The processor microcode seems to be up-to-date. 2026-02-21T11:13:38.194 INFO:teuthology.orchestra.run.trial166.stdout: 2026-02-21T11:13:38.194 INFO:teuthology.orchestra.run.trial166.stdout:No services need to be restarted. 2026-02-21T11:13:38.195 INFO:teuthology.orchestra.run.trial166.stdout: 2026-02-21T11:13:38.195 INFO:teuthology.orchestra.run.trial166.stdout:No containers need to be restarted. 2026-02-21T11:13:38.195 INFO:teuthology.orchestra.run.trial166.stdout: 2026-02-21T11:13:38.195 INFO:teuthology.orchestra.run.trial166.stdout:No user sessions are running outdated binaries. 2026-02-21T11:13:38.195 INFO:teuthology.orchestra.run.trial166.stdout: 2026-02-21T11:13:38.195 INFO:teuthology.orchestra.run.trial166.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-21T11:13:38.987 INFO:teuthology.orchestra.run.trial166.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-21T11:13:38.997 DEBUG:teuthology.parallel:result is None 2026-02-21T11:13:38.998 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:13:39.167 DEBUG:teuthology.orchestra.run.trial166:> dpkg-query -W -f '${Version}' ceph 2026-02-21T11:13:39.177 INFO:teuthology.orchestra.run.trial166.stdout:20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:13:39.178 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5397-gf7273f30-1jammy 2026-02-21T11:13:39.178 INFO:teuthology.task.install:The correct ceph version 20.3.0-5397-gf7273f30-1jammy is installed. 2026-02-21T11:13:39.181 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T11:13:39.181 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:13:39.181 DEBUG:teuthology.orchestra.run.trial166:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T11:13:39.226 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T11:13:39.226 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:13:39.226 DEBUG:teuthology.orchestra.run.trial166:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T11:13:39.273 DEBUG:teuthology.orchestra.run.trial166:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T11:13:39.322 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T11:13:39.322 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:13:39.322 DEBUG:teuthology.orchestra.run.trial166:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T11:13:39.369 DEBUG:teuthology.orchestra.run.trial166:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T11:13:39.418 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T11:13:39.418 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:13:39.418 DEBUG:teuthology.orchestra.run.trial166:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T11:13:39.466 DEBUG:teuthology.orchestra.run.trial166:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T11:13:39.513 INFO:teuthology.run_tasks:Running task workunit... 2026-02-21T11:13:39.542 INFO:tasks.workunit:Pulling workunits from ref f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:13:39.543 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-21T11:13:39.543 INFO:tasks.workunit:timeout=3h 2026-02-21T11:13:39.543 INFO:tasks.workunit:cleanup=True 2026-02-21T11:13:39.544 DEBUG:teuthology.orchestra.run.trial166:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-21T11:13:39.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:13:39.558 INFO:teuthology.orchestra.run.trial166.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-21T11:13:39.559 DEBUG:teuthology.orchestra.run.trial166:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T11:13:39.606 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-21T11:13:39.606 DEBUG:teuthology.orchestra.run.trial166:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-21T11:13:39.650 DEBUG:teuthology.orchestra.run.trial166:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7273f3049146bdd7fa7b4a400114872a50228e9 2026-02-21T11:13:39.700 INFO:tasks.workunit.client.0.trial166.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-21T11:14:01.858 INFO:tasks.workunit.client.0.trial166.stderr:Note: switching to 'f7273f3049146bdd7fa7b4a400114872a50228e9'. 2026-02-21T11:14:01.858 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr:state without impacting any branches by switching back to a branch. 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr: git switch -c 2026-02-21T11:14:01.859 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr:Or undo this operation with: 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr: git switch - 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr: 2026-02-21T11:14:01.860 INFO:tasks.workunit.client.0.trial166.stderr:HEAD is now at f7273f30491 Merge branch 'fs_mirror_daemon_status' of https://github.com/kotreshhr/ceph into wip-bharath21-testing-2026-02-20-1039 2026-02-21T11:14:01.868 DEBUG:teuthology.orchestra.run.trial166:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-21T11:14:01.915 DEBUG:teuthology.orchestra.run.trial166:> set -ex 2026-02-21T11:14:01.916 DEBUG:teuthology.orchestra.run.trial166:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-21T11:14:01.962 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-21T11:14:01.963 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-21T11:14:01.964 DEBUG:teuthology.orchestra.run.trial166:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7273f3049146bdd7fa7b4a400114872a50228e9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-21T11:14:02.013 INFO:tasks.workunit.client.0.trial166.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ TIMEOUT=300 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ MAX_TIMEOUT=15 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ PG_NUM=4 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ TMPDIR=/tmp 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-21T11:14:02.014 INFO:tasks.workunit.client.0.trial166.stderr:+++ uname 2026-02-21T11:14:02.015 INFO:tasks.workunit.client.0.trial166.stderr:++ '[' Linux = FreeBSD ']' 2026-02-21T11:14:02.015 INFO:tasks.workunit.client.0.trial166.stderr:++ SED=sed 2026-02-21T11:14:02.015 INFO:tasks.workunit.client.0.trial166.stderr:++ AWK=awk 2026-02-21T11:14:02.015 INFO:tasks.workunit.client.0.trial166.stderr:+++ stty -a 2026-02-21T11:14:02.015 INFO:tasks.workunit.client.0.trial166.stderr:+++ head -1 2026-02-21T11:14:02.016 INFO:tasks.workunit.client.0.trial166.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-21T11:14:02.016 INFO:tasks.workunit.client.0.trial166.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T11:14:02.017 INFO:tasks.workunit.client.0.trial166.stderr:++ termwidth= 2026-02-21T11:14:02.017 INFO:tasks.workunit.client.0.trial166.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-21T11:14:02.017 INFO:tasks.workunit.client.0.trial166.stderr:++ DIFFCOLOPTS='-y ' 2026-02-21T11:14:02.017 INFO:tasks.workunit.client.0.trial166.stderr:++ KERNCORE=kernel.core_pattern 2026-02-21T11:14:02.017 INFO:tasks.workunit.client.0.trial166.stderr:++ EXTRA_OPTS= 2026-02-21T11:14:02.019 INFO:tasks.workunit.client.0.trial166.stderr:++ test '' = TESTS 2026-02-21T11:14:02.019 INFO:tasks.workunit.client.0.trial166.stderr:++ uname -m 2026-02-21T11:14:02.022 INFO:tasks.workunit.client.0.trial166.stderr:+ arch=x86_64 2026-02-21T11:14:02.022 INFO:tasks.workunit.client.0.trial166.stderr:+ case $arch in 2026-02-21T11:14:02.022 INFO:tasks.workunit.client.0.trial166.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-21T11:14:02.022 INFO:tasks.workunit.client.0.trial166.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:+ main test-erasure-code-plugins 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:+ shift 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:+ shopt -s -o xtrace 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-21T11:14:02.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/test-erasure-code-plugins 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-21T11:14:02.024 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-21T11:14:02.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none ' 2026-02-21T11:14:02.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:02.026 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_blaum_roth_warning 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_profile_warning 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:TEST_preload_no_warning 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:TEST_preload_no_warning_default 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:TEST_preload_warning' 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_blaum_roth_warning td/test-erasure-code-plugins 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:119: TEST_ec_profile_blaum_roth_warning: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:121: TEST_ec_profile_blaum_roth_warning: setup td/test-erasure-code-plugins 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:02.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:02.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:02.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:02.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:02.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:02.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:02.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:02.031 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:02.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:02.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:02.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:02.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:02.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:02.034 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:02.034 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:02.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:02.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.036 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:02.036 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:02.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:02.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:14:02.038 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:14:02.038 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.039 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:14:02.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:14:02.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T11:14:02.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T11:14:02.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:122: TEST_ec_profile_blaum_roth_warning: run_mon td/test-erasure-code-plugins a 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:14:02.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:14:02.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:02.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:02.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:02.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:02.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:14:02.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:14:02.110 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:14:02.110 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:02.110 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:02.110 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:02.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:14:02.166 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:02.167 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.168 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.168 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:02.168 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:02.168 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:123: TEST_ec_profile_blaum_roth_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:14:02.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:14:02.334 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:02.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:02.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:02.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: seq 0 2 2026-02-21T11:14:02.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 0 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:02.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:02.361 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:02.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:14:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:14:02.363 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2ad42521-c30e-4709-87df-2cb71ad863a3 2026-02-21T11:14:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2ad42521-c30e-4709-87df-2cb71ad863a3' 2026-02-21T11:14:02.364 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 2ad42521-c30e-4709-87df-2cb71ad863a3 2026-02-21T11:14:02.364 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:02.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB6k5lpWvoXFhAArDg11uOadeJeul2SN5/Otg== 2026-02-21T11:14:02.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB6k5lpWvoXFhAArDg11uOadeJeul2SN5/Otg=="}' 2026-02-21T11:14:02.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2ad42521-c30e-4709-87df-2cb71ad863a3 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:02.483 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:02.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:02.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB6k5lpWvoXFhAArDg11uOadeJeul2SN5/Otg== --osd-uuid 2ad42521-c30e-4709-87df-2cb71ad863a3 2026-02-21T11:14:02.507 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:02.502+0000 7fd88291ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:02.508 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:02.502+0000 7fd88291ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:02.509 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:02.502+0000 7fd88291ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:02.510 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:02.502+0000 7fd88291ca40 -1 bdev(0x556f9e0de400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:02.510 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:02.506+0000 7fd88291ca40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:14:03.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:14:03.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:03.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:14:03.298 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:14:03.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:03.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:14:03.414 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:14:03.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:14:03.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:03.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:03.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:03.436 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:03.430+0000 7f91c8652a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:03.439 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:03.434+0000 7f91c8652a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:03.440 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:03.434+0000 7f91c8652a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:03.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:03.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:03.956 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:03.950+0000 7f91c8652a40 -1 Falling back to public interface 2026-02-21T11:14:04.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:04.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:04.636 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:04.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:04.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:04.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:04.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:05.043 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:05.038+0000 7f91c8652a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:14:05.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:05.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:05.745 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:05.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:14:05.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:05.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:05.956 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3761405221,v1:127.0.0.1:6801/3761405221] [v2:127.0.0.1:6802/3761405221,v1:127.0.0.1:6803/3761405221] exists,up 2ad42521-c30e-4709-87df-2cb71ad863a3 2026-02-21T11:14:05.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:05.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:05.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:05.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-21T11:14:05.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 1 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-21T11:14:05.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:05.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:14:05.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-21T11:14:05.963 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:05.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=496ed83a-29ba-4b64-a77c-4aee708244e4 2026-02-21T11:14:05.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 496ed83a-29ba-4b64-a77c-4aee708244e4' 2026-02-21T11:14:05.964 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 496ed83a-29ba-4b64-a77c-4aee708244e4 2026-02-21T11:14:05.964 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:05.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB9k5lpXxTcORAAp4vg5Eiq8i144mEcZCoMVg== 2026-02-21T11:14:05.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB9k5lpXxTcORAAp4vg5Eiq8i144mEcZCoMVg=="}' 2026-02-21T11:14:05.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 496ed83a-29ba-4b64-a77c-4aee708244e4 -i td/test-erasure-code-plugins/1/new.json 2026-02-21T11:14:06.196 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:06.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-21T11:14:06.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB9k5lpXxTcORAAp4vg5Eiq8i144mEcZCoMVg== --osd-uuid 496ed83a-29ba-4b64-a77c-4aee708244e4 2026-02-21T11:14:06.222 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:06.218+0000 7f59382b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:06.224 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:06.218+0000 7f59382b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:06.225 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:06.218+0000 7f59382b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:06.225 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:06.218+0000 7f59382b5a40 -1 bdev(0x564d18a78400 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:06.225 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:06.218+0000 7f59382b5a40 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-21T11:14:06.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-21T11:14:06.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:06.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:14:06.886 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:14:06.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:07.177 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:14:07.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:14:07.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:14:07.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:07.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:07.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:07.199 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:07.194+0000 7f52ee56ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:07.201 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:07.194+0000 7f52ee56ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:07.202 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:07.198+0000 7f52ee56ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:07.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:14:07.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:07.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:14:07.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:07.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:07.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:07.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:07.384 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:07.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:07.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:14:07.580 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:07.574+0000 7f52ee56ea40 -1 Falling back to public interface 2026-02-21T11:14:07.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:08.215 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:08.210+0000 7f52ee56ea40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:14:08.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:08.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:08.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:08.599 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:08.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:08.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:14:08.812 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2401140669,v1:127.0.0.1:6811/2401140669] [v2:127.0.0.1:6812/2401140669,v1:127.0.0.1:6813/2401140669] exists,up 496ed83a-29ba-4b64-a77c-4aee708244e4 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 2 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:08.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:08.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:08.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:08.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:08.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:08.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:08.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:08.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:14:08.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-21T11:14:08.820 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:08.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6432b950-c86a-4a00-9527-4f3f39c12380 2026-02-21T11:14:08.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6432b950-c86a-4a00-9527-4f3f39c12380' 2026-02-21T11:14:08.820 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 6432b950-c86a-4a00-9527-4f3f39c12380 2026-02-21T11:14:08.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:08.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCAk5lpB8dSMRAANV0C6XfgRqLHOQsR7BShhw== 2026-02-21T11:14:08.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCAk5lpB8dSMRAANV0C6XfgRqLHOQsR7BShhw=="}' 2026-02-21T11:14:08.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6432b950-c86a-4a00-9527-4f3f39c12380 -i td/test-erasure-code-plugins/2/new.json 2026-02-21T11:14:09.055 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:09.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-21T11:14:09.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAk5lpB8dSMRAANV0C6XfgRqLHOQsR7BShhw== --osd-uuid 6432b950-c86a-4a00-9527-4f3f39c12380 2026-02-21T11:14:09.082 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:09.074+0000 7f294c3c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:09.083 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:09.078+0000 7f294c3c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:09.084 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:09.078+0000 7f294c3c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:09.085 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:09.078+0000 7f294c3c5a40 -1 bdev(0x563bfc742400 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:09.085 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:09.078+0000 7f294c3c5a40 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-21T11:14:09.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-21T11:14:09.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:09.730 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:14:09.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:14:09.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:10.025 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:14:10.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:14:10.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:14:10.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:10.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:10.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:10.048 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:10.042+0000 7f5745214a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:10.049 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:10.042+0000 7f5745214a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:10.051 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:10.046+0000 7f5745214a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:10.235 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:10.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:10.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:10.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:14:10.387 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:10.382+0000 7f5745214a40 -1 Falling back to public interface 2026-02-21T11:14:10.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:11.087 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:11.082+0000 7f5745214a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:14:11.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:11.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:11.449 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:11.450 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:11.450 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:11.450 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:14:11.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:12.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:12.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:12.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:14:12.677 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:12.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:12.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:14:12.885 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1126830451,v1:127.0.0.1:6819/1126830451] [v2:127.0.0.1:6820/1126830451,v1:127.0.0.1:6821/1126830451] exists,up 6432b950-c86a-4a00-9527-4f3f39c12380 2026-02-21T11:14:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:127: TEST_ec_profile_blaum_roth_warning: create_rbd_pool 2026-02-21T11:14:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-21T11:14:13.083 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' does not exist 2026-02-21T11:14:13.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-21T11:14:13.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:14:13.330 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:14:13.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:14:14.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-21T11:14:14.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:128: TEST_ec_profile_blaum_roth_warning: wait_for_clean 2026-02-21T11:14:14.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:14:14.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:14:14.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:14:14.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:14:14.625 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:14:14.625 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:14:14.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:14:14.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:14:14.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:14:14.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:14:14.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:14:14.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:14:14.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:14:14.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:14:14.720 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:14:14.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:14:14.928 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:14:14.928 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:14:14.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:14:14.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:14:14.928 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:14:15.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836484 2026-02-21T11:14:15.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836484 2026-02-21T11:14:15.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484' 2026-02-21T11:14:15.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:14:15.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:14:15.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705668 2026-02-21T11:14:15.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705668 2026-02-21T11:14:15.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705668' 2026-02-21T11:14:15.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:14:15.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:14:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574851 2026-02-21T11:14:15.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574851 2026-02-21T11:14:15.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-55834574851' 2026-02-21T11:14:15.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:14:15.163 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836484 2026-02-21T11:14:15.163 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:14:15.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:14:15.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836484 2026-02-21T11:14:15.166 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:14:15.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836484 2026-02-21T11:14:15.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-21T11:14:15.167 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836484 2026-02-21T11:14:15.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:14:15.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-21T11:14:15.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:14:16.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:14:16.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:14:16.606 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-21T11:14:16.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:14:17.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:14:17.608 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:14:17.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-21T11:14:17.815 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:14:17.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705668 2026-02-21T11:14:17.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:14:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:14:17.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705668 2026-02-21T11:14:17.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:14:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705668 2026-02-21T11:14:17.819 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 38654705668 2026-02-21T11:14:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-21T11:14:17.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:14:18.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-21T11:14:18.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:14:18.037 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574851 2026-02-21T11:14:18.038 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:14:18.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:14:18.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574851 2026-02-21T11:14:18.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:14:18.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574851 2026-02-21T11:14:18.041 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 55834574851 2026-02-21T11:14:18.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-21T11:14:18.041 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:14:18.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-21T11:14:18.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:14:18.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:14:18.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:14:18.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:14:18.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-21T11:14:18.744 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:14:18.745 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:14:18.745 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:14:19.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-21T11:14:19.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:14:19.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:14:19.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:130: TEST_ec_profile_blaum_roth_warning: echo 'Starting test for blaum-roth profile health warning' 2026-02-21T11:14:19.020 INFO:tasks.workunit.client.0.trial166.stdout:Starting test for blaum-roth profile health warning 2026-02-21T11:14:19.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:133: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set prof- plugin=jerasure k=3 m=1 technique=blaum_roth w=7 --yes-i-really-mean-it --force 2026-02-21T11:14:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:135: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set test crush-device-class= crush-failure-domain=osd crush-num-failure-domains=0 crush-osds-per-failure-domain=0 crush-root=default k=4 l=3 m=2 name=lrcprofile plugin=lrc 2026-02-21T11:14:19.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: get_asok_path mon.a 2026-02-21T11:14:19.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:19.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:19.568 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:19.568 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:19.569 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:19.569 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:19.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: CEPH_ARGS= 2026-02-21T11:14:19.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:19.619 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:19.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:137: TEST_ec_profile_blaum_roth_warning: sleep 10 2026-02-21T11:14:29.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:138: TEST_ec_profile_blaum_roth_warning: grep -F '1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:29.630 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:22.106+0000 7fc473e9b640 0 log_channel(cluster) log [WRN] : Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-21T11:14:29.630 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:22.106+0000 7fc471696640 10 mon.a@0(leader).log v82 logging 2026-02-21T11:14:22.108316+0000 mon.a (mon.0) 152 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-21T11:14:29.630 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:22.158+0000 7fc46ee91640 7 mon.a@0(leader).log v83 update_from_paxos applying incremental log 83 2026-02-21T11:14:22.108316+0000 mon.a (mon.0) 152 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.631 INFO:tasks.workunit.client.0.trial166.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-21T11:14:29.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:139: TEST_ec_profile_blaum_roth_warning: grep -F 'w+1=8 for the EC profile prof- is not prime and could lead to data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:29.633 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.633 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.634 INFO:tasks.workunit.client.0.trial166.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-21T11:14:29.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:141: TEST_ec_profile_blaum_roth_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:14:29.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:29.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:29.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:29.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:29.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:29.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:29.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:29.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:29.743 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:29.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:29.744 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:29.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:29.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:29.746 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:29.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:29.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:29.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:29.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:29.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:29.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:29.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:29.757 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:29.757 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:29.757 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:29.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:142: TEST_ec_profile_blaum_roth_warning: return 0 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:29.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:29.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:29.762 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:29.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:29.763 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:29.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:29.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:29.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:29.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:29.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:29.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:29.767 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:29.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:29.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:29.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:29.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:29.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:29.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:29.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:29.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:29.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:29.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:14:29.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:14:29.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:29.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:29.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:14:29.774 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:14:29.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.796 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:14:29.796 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:29.796 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:29.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:29.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:29.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:29.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:29.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:14:29.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:14:29.827 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:14:29.827 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:29.827 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:29.827 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:14:29.827 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:29.828 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:29.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:29.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:14:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:14:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:14:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:29.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:14:29.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:30.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:30.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:30.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:30.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-21T11:14:30.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-21T11:14:30.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-21T11:14:30.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:30.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:30.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:30.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:30.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:14:30.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:14:30.093 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:30.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=45a9cad5-f912-46f7-a47f-58a84f30be1e 2026-02-21T11:14:30.093 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 45a9cad5-f912-46f7-a47f-58a84f30be1e 2026-02-21T11:14:30.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 45a9cad5-f912-46f7-a47f-58a84f30be1e' 2026-02-21T11:14:30.094 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:30.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCWk5lpuf8JBhAA0cDne/puTdoVKlYZuNkyjA== 2026-02-21T11:14:30.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCWk5lpuf8JBhAA0cDne/puTdoVKlYZuNkyjA=="}' 2026-02-21T11:14:30.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 45a9cad5-f912-46f7-a47f-58a84f30be1e -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:30.212 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:30.220 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:30.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWk5lpuf8JBhAA0cDne/puTdoVKlYZuNkyjA== --osd-uuid 45a9cad5-f912-46f7-a47f-58a84f30be1e 2026-02-21T11:14:30.236 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:30.230+0000 7f83247eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:30.237 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:30.230+0000 7f83247eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:30.238 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:30.234+0000 7f83247eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:30.239 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:30.234+0000 7f83247eca40 -1 bdev(0x556f974a8400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:30.239 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:30.234+0000 7f83247eca40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:14:30.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:14:30.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:30.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:14:30.931 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:14:30.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:31.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:14:31.042 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:14:31.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:14:31.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:31.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:31.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:31.055 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:31.050+0000 7fc18e0afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:31.057 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:31.050+0000 7fc18e0afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:31.058 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:31.054+0000 7fc18e0afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:31.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:14:31.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:31.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:14:31.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:31.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:31.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:31.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:31.154 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:31.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:31.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:31.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:31.440 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:31.434+0000 7fc18e0afa40 -1 Falling back to public interface 2026-02-21T11:14:32.142 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:32.138+0000 7fc18e0afa40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:14:32.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:32.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:32.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:32.259 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:32.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:32.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:32.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:33.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:33.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:33.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:14:33.406 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:33.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:33.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:33.617 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2522599591,v1:127.0.0.1:6801/2522599591] [v2:127.0.0.1:6802/2522599591,v1:127.0.0.1:6803/2522599591] exists,up 45a9cad5-f912-46f7-a47f-58a84f30be1e 2026-02-21T11:14:33.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:33.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:33.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:33.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-21T11:14:33.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:33.621 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-21T11:14:33.622 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:33.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=636e1dea-81d8-40f5-b80f-9bd45ad7f584 2026-02-21T11:14:33.623 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 636e1dea-81d8-40f5-b80f-9bd45ad7f584 2026-02-21T11:14:33.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 636e1dea-81d8-40f5-b80f-9bd45ad7f584' 2026-02-21T11:14:33.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:33.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZk5lpHhygJRAAm0Qp9S8d5VFOwhQIoXyqbw== 2026-02-21T11:14:33.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZk5lpHhygJRAAm0Qp9S8d5VFOwhQIoXyqbw=="}' 2026-02-21T11:14:33.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 636e1dea-81d8-40f5-b80f-9bd45ad7f584 -i td/test-erasure-code-plugins/1/new.json 2026-02-21T11:14:33.849 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:33.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-21T11:14:33.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZk5lpHhygJRAAm0Qp9S8d5VFOwhQIoXyqbw== --osd-uuid 636e1dea-81d8-40f5-b80f-9bd45ad7f584 2026-02-21T11:14:33.883 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:33.878+0000 7ff37414ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:33.885 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:33.878+0000 7ff37414ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:33.886 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:33.882+0000 7ff37414ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:33.886 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:33.882+0000 7ff37414ba40 -1 bdev(0x55c1101b6400 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:33.886 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:33.882+0000 7ff37414ba40 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-21T11:14:34.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-21T11:14:34.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:34.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:14:34.527 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:14:34.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:34.815 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:14:34.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:14:34.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:14:34.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:34.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:34.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:34.837 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:34.830+0000 7f0a6d5e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:34.839 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:34.834+0000 7f0a6d5e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:34.840 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:34.834+0000 7f0a6d5e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:35.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:14:35.160 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:35.154+0000 7f0a6d5e7a40 -1 Falling back to public interface 2026-02-21T11:14:35.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:35.804 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:35.798+0000 7f0a6d5e7a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:14:36.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:36.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:36.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:36.235 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:36.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:36.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:14:36.440 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/907326458,v1:127.0.0.1:6811/907326458] [v2:127.0.0.1:6812/907326458,v1:127.0.0.1:6813/907326458] exists,up 636e1dea-81d8-40f5-b80f-9bd45ad7f584 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:36.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:36.445 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:36.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:14:36.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-21T11:14:36.448 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:36.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d0f88158-dfe6-4048-ab5c-fbbeb30c38fd 2026-02-21T11:14:36.448 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 d0f88158-dfe6-4048-ab5c-fbbeb30c38fd 2026-02-21T11:14:36.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d0f88158-dfe6-4048-ab5c-fbbeb30c38fd' 2026-02-21T11:14:36.449 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:36.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCck5lpKOsdGxAAuHzqeJFxDZdNeJ8F5PiLnw== 2026-02-21T11:14:36.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCck5lpKOsdGxAAuHzqeJFxDZdNeJ8F5PiLnw=="}' 2026-02-21T11:14:36.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d0f88158-dfe6-4048-ab5c-fbbeb30c38fd -i td/test-erasure-code-plugins/2/new.json 2026-02-21T11:14:36.697 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:36.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-21T11:14:36.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCck5lpKOsdGxAAuHzqeJFxDZdNeJ8F5PiLnw== --osd-uuid d0f88158-dfe6-4048-ab5c-fbbeb30c38fd 2026-02-21T11:14:36.724 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:36.718+0000 7f8f53014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:36.726 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:36.718+0000 7f8f53014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:36.727 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:36.722+0000 7f8f53014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:36.727 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:36.722+0000 7f8f53014a40 -1 bdev(0x560f63c06400 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:36.728 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:36.722+0000 7f8f53014a40 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-21T11:14:37.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-21T11:14:37.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:37.357 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:14:37.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:14:37.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:37.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:14:37.645 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:14:37.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:14:37.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:37.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:37.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:37.667 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:37.662+0000 7fd5c5957a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:37.669 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:37.662+0000 7fd5c5957a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:37.670 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:37.666+0000 7fd5c5957a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:37.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:14:37.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:37.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:14:37.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:37.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:37.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:37.859 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:37.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:37.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:37.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:14:38.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:38.107 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:38.102+0000 7fd5c5957a40 -1 Falling back to public interface 2026-02-21T11:14:38.674 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:38.666+0000 7fd5c5957a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:14:39.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:39.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:39.071 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:39.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:39.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:39.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:14:39.275 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2421989380,v1:127.0.0.1:6819/2421989380] [v2:127.0.0.1:6820/2421989380,v1:127.0.0.1:6821/2421989380] exists,up d0f88158-dfe6-4048-ab5c-fbbeb30c38fd 2026-02-21T11:14:39.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:39.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:39.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:39.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-21T11:14:39.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-21T11:14:39.481 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' does not exist 2026-02-21T11:14:39.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-21T11:14:39.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:14:39.769 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:14:39.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:14:40.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-21T11:14:41.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-21T11:14:41.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:14:41.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:14:41.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:14:41.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:14:41.060 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:14:41.060 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:14:41.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:14:41.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:14:41.061 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:14:41.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:14:41.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:14:41.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:14:41.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:14:41.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:14:41.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:14:41.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:14:41.355 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:14:41.355 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:14:41.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:14:41.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:14:41.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:14:41.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836484 2026-02-21T11:14:41.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836484 2026-02-21T11:14:41.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484' 2026-02-21T11:14:41.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:14:41.434 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:14:41.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705667 2026-02-21T11:14:41.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705667 2026-02-21T11:14:41.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-21T11:14:41.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:14:41.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:14:41.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574851 2026-02-21T11:14:41.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574851 2026-02-21T11:14:41.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-55834574851' 2026-02-21T11:14:41.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:14:41.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836484 2026-02-21T11:14:41.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:14:41.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:14:41.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836484 2026-02-21T11:14:41.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:14:41.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836484 2026-02-21T11:14:41.593 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836484 2026-02-21T11:14:41.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-21T11:14:41.594 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:14:41.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-21T11:14:41.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:14:42.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:14:42.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:14:43.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-21T11:14:43.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:14:43.026 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705667 2026-02-21T11:14:43.026 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:14:43.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:14:43.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705667 2026-02-21T11:14:43.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:14:43.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705667 2026-02-21T11:14:43.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-21T11:14:43.029 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 38654705667 2026-02-21T11:14:43.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:14:43.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-21T11:14:43.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:14:43.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574851 2026-02-21T11:14:43.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:14:43.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:14:43.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574851 2026-02-21T11:14:43.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:14:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574851 2026-02-21T11:14:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-21T11:14:43.263 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 55834574851 2026-02-21T11:14:43.263 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:14:43.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-21T11:14:43.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:14:43.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:14:43.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:14:43.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-21T11:14:43.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:14:43.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:14:43.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:14:43.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:14:43.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:14:43.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:14:43.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:14:43.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-21T11:14:43.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:14:43.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:14:43.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:14:44.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-21T11:14:44.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:14:44.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:14:44.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-21T11:14:44.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-21T11:14:44.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-21T11:14:44.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:44.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:44.517 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:44.517 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:44.517 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:44.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:44.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-21T11:14:44.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:44.565 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:44.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:44.573 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:44.389+0000 7fd86c1ba640 0 mon.a@0(leader).osd e19 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:44.574 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:44.497+0000 7fd86c1ba640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:44.574 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:44.497+0000 7fd86c1ba640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:44.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-21T11:14:44.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-21T11:14:44.839 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-21T11:14:44.839 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:44.839 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:44.840 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:44.840 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:44.840 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:44.840 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:44.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-21T11:14:44.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:44.888 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:44.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:44.897 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:44.713+0000 7fd86c1ba640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:44.897 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:44.821+0000 7fd86c1ba640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:44.897 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:44.821+0000 7fd86c1ba640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:44.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-21T11:14:44.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-21T11:14:45.169 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-21T11:14:45.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:45.217 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:45.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:45.224 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.045+0000 7fd86c1ba640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:45.225 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.153+0000 7fd86c1ba640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:45.225 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.153+0000 7fd86c1ba640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-21T11:14:45.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-21T11:14:45.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-21T11:14:45.487 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-21T11:14:45.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:45.537 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:45.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:45.544 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.365+0000 7fd86c1ba640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-21T11:14:45.544 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.469+0000 7fd86c1ba640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-21T11:14:45.545 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.469+0000 7fd86c1ba640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-21T11:14:45.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-21T11:14:45.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-21T11:14:45.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-21T11:14:45.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-21T11:14:45.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:45.857 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:45.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:45.865 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.685+0000 7fd86c1ba640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-21T11:14:45.865 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.789+0000 7fd86c1ba640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-21T11:14:45.865 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:45.789+0000 7fd86c1ba640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-21T11:14:45.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-21T11:14:45.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-21T11:14:46.132 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-21T11:14:46.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:46.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:46.133 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:46.133 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.133 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:46.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-21T11:14:46.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:46.180 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:46.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:46.189 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:46.009+0000 7fd86c1ba640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-21T11:14:46.190 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:46.113+0000 7fd86c1ba640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-21T11:14:46.190 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:14:46.113+0000 7fd86c1ba640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:46.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:46.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:46.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:46.299 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:46.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:46.300 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:46.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:46.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:46.302 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:46.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:46.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:46.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:46.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:46.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:46.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:46.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:46.315 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:46.315 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.315 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-21T11:14:46.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:46.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:46.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:46.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:46.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:46.322 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:46.322 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:46.323 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:46.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:46.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:46.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:46.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:46.325 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:46.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:46.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:46.327 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:46.328 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.328 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:46.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:46.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:46.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:14:46.330 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:14:46.330 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:14:46.332 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:14:46.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:14:46.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:14:46.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:14:46.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-21T11:14:46.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:14:46.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:14:46.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:14:46.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:14:46.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-21T11:14:46.354 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:14:46.354 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:46.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:46.355 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:46.355 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.355 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:46.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-21T11:14:46.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:14:46.381 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:14:46.382 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:46.382 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:46.382 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:14:46.383 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:14:46.383 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:46.383 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:46.384 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:14:46.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:14:46.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:46.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:46.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:14:46.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:46.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:46.457 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:14:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:14:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:14:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:14:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:14:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:14:46.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:14:46.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:14:46.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:46.617 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:46.617 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:46.617 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.617 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.617 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:46.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:46.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:46.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:46.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:46.695 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:46.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-21T11:14:46.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:46.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:46.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:46.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-21T11:14:46.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:14:46.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:46.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56abc6b3-5e81-447b-9ae6-e46f4776b98d 2026-02-21T11:14:46.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 56abc6b3-5e81-447b-9ae6-e46f4776b98d' 2026-02-21T11:14:46.708 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 56abc6b3-5e81-447b-9ae6-e46f4776b98d 2026-02-21T11:14:46.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:46.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmk5lp+2q0KhAA0ExbW7GVI6w9mSXXslL8eg== 2026-02-21T11:14:46.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmk5lp+2q0KhAA0ExbW7GVI6w9mSXXslL8eg=="}' 2026-02-21T11:14:46.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56abc6b3-5e81-447b-9ae6-e46f4776b98d -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:46.831 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:46.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:46.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQCmk5lp+2q0KhAA0ExbW7GVI6w9mSXXslL8eg== --osd-uuid 56abc6b3-5e81-447b-9ae6-e46f4776b98d 2026-02-21T11:14:46.856 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:46.849+0000 7fbe32c1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:46.857 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:46.849+0000 7fbe32c1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:46.858 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:46.853+0000 7fbe32c1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:46.858 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:46.853+0000 7fbe32c1ea40 -1 bdev(0x564261036400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:46.859 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:46.853+0000 7fbe32c1ea40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:14:47.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:14:47.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:47.508 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:14:47.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:14:47.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:14:47.622 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:14:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-21T11:14:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:47.636 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:47.629+0000 7fce90452a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:47.638 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:47.629+0000 7fce90452a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:47.639 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:47.633+0000 7fce90452a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:47.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:14:47.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:47.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:14:47.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:47.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:47.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:47.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:47.735 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:47.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:47.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:47.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:48.180 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:48.177+0000 7fce90452a40 -1 Falling back to public interface 2026-02-21T11:14:48.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:48.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:48.840 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:48.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:48.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:48.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:48.944 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:49.127 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:49.125+0000 7fce90452a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:14:49.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:49.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:49.946 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:49.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:14:49.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:49.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:50.156 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/806250203,v1:127.0.0.1:6801/806250203] [v2:127.0.0.1:6802/806250203,v1:127.0.0.1:6803/806250203] exists,up 56abc6b3-5e81-447b-9ae6-e46f4776b98d 2026-02-21T11:14:50.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:50.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:50.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:50.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-21T11:14:50.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:50.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:14:50.221 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:50.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:50.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:14:50.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:14:50.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:50.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:50.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:50.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:50.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:50.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:50.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:50.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:50.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:50.340 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:50.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:50.341 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:50.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:50.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:50.342 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:50.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:50.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:50.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:50.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:50.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:50.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:50.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:50.350 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:50.350 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.350 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:50.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:50.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:50.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-21T11:14:50.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:50.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:50.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:50.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:50.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:50.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:50.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:50.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:50.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:50.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:50.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:50.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:50.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:50.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:50.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:50.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:50.362 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:50.362 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.362 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:50.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:14:50.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:14:50.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:14:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:14:50.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:14:50.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-21T11:14:50.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:14:50.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:50.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:50.388 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:50.388 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.388 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.388 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:50.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-21T11:14:50.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:50.418 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:50.419 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:50.419 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.419 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.420 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:50.420 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:50.420 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:14:50.473 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.474 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.475 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:50.475 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:50.475 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:14:50.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:14:50.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:50.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:14:50.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:14:50.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:14:50.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:14:50.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:14:50.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:14:50.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:50.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:50.634 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:50.634 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.634 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:50.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:50.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:50.653 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-21T11:14:50.653 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:50.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:50.702 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:50.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-21T11:14:50.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:50.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:50.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:50.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:14:50.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:50.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b21f0d54-c963-4b64-8384-65cbb6140a40 2026-02-21T11:14:50.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b21f0d54-c963-4b64-8384-65cbb6140a40' 2026-02-21T11:14:50.715 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 b21f0d54-c963-4b64-8384-65cbb6140a40 2026-02-21T11:14:50.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:50.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCqk5lpKWP9KhAAnt1+CQu9ZMs1hOnamGZbmw== 2026-02-21T11:14:50.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCqk5lpKWP9KhAAnt1+CQu9ZMs1hOnamGZbmw=="}' 2026-02-21T11:14:50.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b21f0d54-c963-4b64-8384-65cbb6140a40 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:50.833 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:50.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:50.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQCqk5lpKWP9KhAAnt1+CQu9ZMs1hOnamGZbmw== --osd-uuid b21f0d54-c963-4b64-8384-65cbb6140a40 2026-02-21T11:14:50.856 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:50.853+0000 7f900f6d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:50.858 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:50.853+0000 7f900f6d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:50.859 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:50.857+0000 7f900f6d2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:50.859 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:50.857+0000 7f900f6d2a40 -1 bdev(0x55d01e9a2400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:50.860 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:50.857+0000 7f900f6d2a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:14:51.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:14:51.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:51.582 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:14:51.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:14:51.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:51.702 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:14:51.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:14:51.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-21T11:14:51.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:51.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:51.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:51.725 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:51.721+0000 7f0ed480da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:51.727 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:51.725+0000 7f0ed480da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:51.729 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:51.725+0000 7f0ed480da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:51.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:14:51.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:51.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:14:51.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:51.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:51.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:51.811 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:51.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:51.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:51.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:51.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:52.200 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:52.197+0000 7f0ed480da40 -1 Falling back to public interface 2026-02-21T11:14:52.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:52.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:52.919 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:52.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:52.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:52.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:53.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:53.193 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:53.189+0000 7f0ed480da40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:14:54.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:54.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:54.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:14:54.025 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:54.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:54.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:54.233 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2955148204,v1:127.0.0.1:6801/2955148204] [v2:127.0.0.1:6802/2955148204,v1:127.0.0.1:6803/2955148204] exists,up b21f0d54-c963-4b64-8384-65cbb6140a40 2026-02-21T11:14:54.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:54.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:14:54.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:54.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:14:54.295 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:54.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:54.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:14:54.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:14:54.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:54.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:54.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:54.305 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:54.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:54.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:54.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:54.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:54.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:54.413 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:54.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:54.414 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:54.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:54.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:54.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:54.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:54.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:54.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:54.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:54.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:54.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:54.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:54.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:54.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:54.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:54.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:54.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-21T11:14:54.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-21T11:14:54.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:54.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:14:54.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:54.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:54.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:54.427 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:54.428 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:54.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:54.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:54.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:54.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:54.430 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:54.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:54.430 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:54.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:54.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:54.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:54.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:54.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:54.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:54.432 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:54.433 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:54.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:54.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:54.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:54.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:54.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:54.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:54.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:14:54.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:14:54.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:14:54.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:14:54.441 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:14:54.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-21T11:14:54.463 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:14:54.463 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:54.463 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:54.464 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:54.464 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.464 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:54.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-21T11:14:54.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:14:54.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:14:54.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:54.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:54.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:54.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:54.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:14:54.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:14:54.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:54.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:54.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:14:54.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:54.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:54.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:14:54.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:14:54.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:14:54.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:54.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:54.713 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:54.713 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.713 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:54.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:54.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:54.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-21T11:14:54.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:54.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:54.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:54.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.732 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.732 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:54.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:54.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:54.784 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:54.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-21T11:14:54.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:54.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:54.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:54.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-21T11:14:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:14:54.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=19a5712b-04e1-49f2-b381-b4e95089356d 2026-02-21T11:14:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 19a5712b-04e1-49f2-b381-b4e95089356d' 2026-02-21T11:14:54.797 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 19a5712b-04e1-49f2-b381-b4e95089356d 2026-02-21T11:14:54.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:54.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCuk5lprDkKMBAAxaHLO+lkW6TQ85qdwsgyEQ== 2026-02-21T11:14:54.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCuk5lprDkKMBAAxaHLO+lkW6TQ85qdwsgyEQ=="}' 2026-02-21T11:14:54.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 19a5712b-04e1-49f2-b381-b4e95089356d -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:54.921 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:54.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:54.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQCuk5lprDkKMBAAxaHLO+lkW6TQ85qdwsgyEQ== --osd-uuid 19a5712b-04e1-49f2-b381-b4e95089356d 2026-02-21T11:14:54.947 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:54.945+0000 7faafb394a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:54.949 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:54.945+0000 7faafb394a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:54.950 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:54.949+0000 7faafb394a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:54.951 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:54.949+0000 7faafb394a40 -1 bdev(0x564ffffa0400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:54.951 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:54.949+0000 7faafb394a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:14:55.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:14:55.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:14:55.826 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:14:55.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:14:55.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:14:55.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:14:55.938 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:14:55.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-21T11:14:55.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:14:55.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:14:55.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:14:55.962 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:55.961+0000 7f3dde483a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:55.965 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:55.961+0000 7f3dde483a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:55.967 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:55.965+0000 7f3dde483a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:56.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:14:56.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:14:56.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:14:56.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:14:56.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:14:56.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:56.047 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:56.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:14:56.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:56.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:56.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:56.356 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:56.353+0000 7f3dde483a40 -1 Falling back to public interface 2026-02-21T11:14:57.067 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:57.065+0000 7f3dde483a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:14:57.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:57.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:57.162 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:14:57.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:14:57.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:57.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:57.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:14:58.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:14:58.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:14:58.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:14:58.372 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:14:58.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:14:58.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:14:58.580 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2748681964,v1:127.0.0.1:6801/2748681964] [v2:127.0.0.1:6802/2748681964,v1:127.0.0.1:6803/2748681964] exists,up 19a5712b-04e1-49f2-b381-b4e95089356d 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.581 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:14:58.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:58.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:14:58.637 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:14:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:14:58.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:14:58.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:58.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:58.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:58.647 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:58.647 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:58.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:58.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:58.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:58.755 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:58.756 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:58.756 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:58.757 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:58.758 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:58.758 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:58.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:58.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:58.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:58.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:58.760 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:58.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:58.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:58.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:58.767 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:58.767 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.767 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:58.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:58.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:58.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-21T11:14:58.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:14:58.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:14:58.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:14:58.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:14:58.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:14:58.772 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:14:58.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:14:58.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:14:58.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:14:58.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:58.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:14:58.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:14:58.775 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:14:58.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:14:58.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:14:58.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:14:58.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:14:58.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:14:58.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:14:58.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:14:58.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:14:58.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:14:58.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:14:58.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:14:58.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:14:58.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:14:58.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:14:58.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-21T11:14:58.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:58.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:14:58.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:14:58.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:14:58.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:14:58.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-21T11:14:58.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:14:58.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:58.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:58.806 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:58.806 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.806 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:58.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-21T11:14:58.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:58.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:58.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:58.890 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:58.891 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:58.891 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:58.891 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:58.891 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:14:58.891 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:14:58.944 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:14:58.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:58.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:14:58.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:14:58.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:14:58.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:14:58.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:14:59.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:14:59.059 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:59.059 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:59.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:59.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:59.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:59.059 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:59.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:59.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:14:59.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-21T11:14:59.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:14:59.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:14:59.078 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:14:59.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:59.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:59.079 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:14:59.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:14:59.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:14:59.133 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:14:59.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-21T11:14:59.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:14:59.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:14:59.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:14:59.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:14:59.146 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:14:59.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f3309268-8f8b-4772-b707-58ece4fa593e 2026-02-21T11:14:59.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f3309268-8f8b-4772-b707-58ece4fa593e' 2026-02-21T11:14:59.147 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 f3309268-8f8b-4772-b707-58ece4fa593e 2026-02-21T11:14:59.148 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:14:59.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzk5lpFHMoCRAAUCnXbK5Gu0AYlEVq+r8W9A== 2026-02-21T11:14:59.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzk5lpFHMoCRAAUCnXbK5Gu0AYlEVq+r8W9A=="}' 2026-02-21T11:14:59.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f3309268-8f8b-4772-b707-58ece4fa593e -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:59.264 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:14:59.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:14:59.272 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQCzk5lpFHMoCRAAUCnXbK5Gu0AYlEVq+r8W9A== --osd-uuid f3309268-8f8b-4772-b707-58ece4fa593e 2026-02-21T11:14:59.287 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:59.285+0000 7ff3372b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:59.289 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:59.285+0000 7ff3372b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:59.290 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:59.289+0000 7ff3372b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:14:59.291 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:59.289+0000 7ff3372b2a40 -1 bdev(0x5569d1f1e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:14:59.291 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:14:59.289+0000 7ff3372b2a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:00.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:00.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:00.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:00.030 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:00.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:00.153 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:00.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:00.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-21T11:15:00.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:00.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:00.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:00.175 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:00.173+0000 7f9d9ebbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:00.177 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:00.173+0000 7f9d9ebbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:00.178 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:00.177+0000 7f9d9ebbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:00.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:00.261 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:00.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:00.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:00.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:00.692 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:00.689+0000 7f9d9ebbfa40 -1 Falling back to public interface 2026-02-21T11:15:01.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:01.373 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:01.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:01.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:01.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:01.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:01.408 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:01.405+0000 7f9d9ebbfa40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:01.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:02.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:02.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:02.586 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:02.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:02.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:02.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:02.798 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3579151424,v1:127.0.0.1:6801/3579151424] [v2:127.0.0.1:6802/3579151424,v1:127.0.0.1:6803/3579151424] exists,up f3309268-8f8b-4772-b707-58ece4fa593e 2026-02-21T11:15:02.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:02.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:02.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:02.799 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-21T11:15:02.800 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:02.800 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:02.801 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:02.801 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:02.801 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:02.801 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:02.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-21T11:15:02.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:02.866 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:02.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:02.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:02.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:02.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:02.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:02.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:02.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:02.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:02.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:02.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:02.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:02.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:02.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:02.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:02.986 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:02.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:02.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:02.988 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:02.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:02.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:02.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:02.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:02.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:02.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:02.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:02.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:02.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:02.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:02.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:02.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:03.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:03.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:03.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:03.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:03.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:03.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:03.004 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:03.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:03.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:03.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:03.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:03.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:03.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:03.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:03.009 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:03.009 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.009 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:03.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:03.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:03.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:03.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:03.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:03.013 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:03.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:03.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-21T11:15:03.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:03.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:03.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:03.035 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:03.035 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.035 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:03.035 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:15:03.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:03.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:03.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:03.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:03.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.067 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.068 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:03.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:03.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:03.124 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:03.125 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:03.126 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:03.126 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.126 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.127 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:03.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:03.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:03.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-21T11:15:03.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:03.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:03.183 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:03.183 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.183 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:03.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-21T11:15:03.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:03.240 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.369 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:03.370 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:03.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:03.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-21T11:15:03.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:03.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:03.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:03.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:03.393 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:03.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:03.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:03.396 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:03.396 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7bf8b18-1b08-4774-b641-67088ac292d7 2026-02-21T11:15:03.396 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c7bf8b18-1b08-4774-b641-67088ac292d7' 2026-02-21T11:15:03.397 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 c7bf8b18-1b08-4774-b641-67088ac292d7 2026-02-21T11:15:03.397 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:03.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC3k5lprLMDGBAAEpgzUiHWFp1Lqvl+E/PFGA== 2026-02-21T11:15:03.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC3k5lprLMDGBAAEpgzUiHWFp1Lqvl+E/PFGA=="}' 2026-02-21T11:15:03.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7bf8b18-1b08-4774-b641-67088ac292d7 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:03.514 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:03.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:03.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC3k5lprLMDGBAAEpgzUiHWFp1Lqvl+E/PFGA== --osd-uuid c7bf8b18-1b08-4774-b641-67088ac292d7 2026-02-21T11:15:03.538 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:03.537+0000 7f65535f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:03.539 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:03.537+0000 7f65535f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:03.540 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:03.537+0000 7f65535f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:03.541 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:03.537+0000 7f65535f2a40 -1 bdev(0x561db2d60400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:03.541 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:03.537+0000 7f65535f2a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:04.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:04.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:04.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:04.315 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:04.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:04.435 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:04.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:04.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:04.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:04.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:04.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:04.451 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:04.449+0000 7f02bfa32a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:04.452 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:04.449+0000 7f02bfa32a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:04.453 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:04.449+0000 7f02bfa32a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:04.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:04.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:04.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:04.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:04.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:04.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:04.559 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:04.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:04.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:04.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:04.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:04.968 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:04.965+0000 7f02bfa32a40 -1 Falling back to public interface 2026-02-21T11:15:05.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:05.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:05.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:05.670 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:05.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:05.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:05.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:05.915 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:05.913+0000 7f02bfa32a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:06.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:06.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:06.879 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:06.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:06.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:06.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:07.099 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3051993592,v1:127.0.0.1:6801/3051993592] [v2:127.0.0.1:6802/3051993592,v1:127.0.0.1:6803/3051993592] exists,up c7bf8b18-1b08-4774-b641-67088ac292d7 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:07.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-21T11:15:07.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:07.156 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:07.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:07.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:07.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:07.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:07.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:07.273 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:07.273 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:07.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:07.274 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:07.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:07.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:07.276 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:07.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:07.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:07.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:07.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:07.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:07.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:07.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:07.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:07.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:07.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:07.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:07.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:07.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:07.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:07.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:07.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:07.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:07.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:07.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:07.288 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:07.290 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:07.290 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:07.290 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:07.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:07.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:07.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:07.292 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:07.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:07.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:07.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:07.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:07.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:07.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:07.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:07.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:07.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:07.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:07.300 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:07.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:07.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:07.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:07.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-21T11:15:07.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:07.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:07.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:07.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:07.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-21T11:15:07.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:07.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:07.322 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:07.322 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:07.323 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.323 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.323 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-21T11:15:07.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:07.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:07.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:07.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:07.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:07.350 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:07.350 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:07.350 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:07.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:07.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:07.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:07.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:07.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:07.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:07.410 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:07.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:07.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:07.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:07.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:07.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:07.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:07.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.579 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:07.580 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:07.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:07.598 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-21T11:15:07.599 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:07.599 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:07.599 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:07.599 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.599 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.599 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:07.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:07.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:07.657 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:07.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-21T11:15:07.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:07.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:07.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:07.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-21T11:15:07.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:07.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:07.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9951ed66-ce27-4417-a9c3-080a47cd268e 2026-02-21T11:15:07.670 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 9951ed66-ce27-4417-a9c3-080a47cd268e 2026-02-21T11:15:07.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9951ed66-ce27-4417-a9c3-080a47cd268e' 2026-02-21T11:15:07.671 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:07.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7k5lphD1kKBAASp2161QqpIl6W4HDKMCC7Q== 2026-02-21T11:15:07.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7k5lphD1kKBAASp2161QqpIl6W4HDKMCC7Q=="}' 2026-02-21T11:15:07.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9951ed66-ce27-4417-a9c3-080a47cd268e -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:07.787 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:07.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:07.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQC7k5lphD1kKBAASp2161QqpIl6W4HDKMCC7Q== --osd-uuid 9951ed66-ce27-4417-a9c3-080a47cd268e 2026-02-21T11:15:07.811 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:07.809+0000 7f8cfa609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:07.812 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:07.809+0000 7f8cfa609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:07.813 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:07.809+0000 7f8cfa609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:07.814 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:07.813+0000 7f8cfa609a40 -1 bdev(0x55968b2a4400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:07.814 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:07.813+0000 7f8cfa609a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:08.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:08.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:08.526 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:08.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:08.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:08.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:08.635 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:08.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-21T11:15:08.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:08.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:08.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:08.650 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:08.645+0000 7fac0cdbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:08.652 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:08.649+0000 7fac0cdbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:08.652 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:08.649+0000 7fac0cdbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:08.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:08.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:08.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:08.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:08.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:08.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:08.750 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:08.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:08.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:08.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:08.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:09.048 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:09.045+0000 7fac0cdbfa40 -1 Falling back to public interface 2026-02-21T11:15:09.690 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:09.689+0000 7fac0cdbfa40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:09.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:09.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:09.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:09.855 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:09.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:09.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:09.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:10.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:10.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:10.979 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:10.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:10.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:10.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:11.199 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2954588696,v1:127.0.0.1:6801/2954588696] [v2:127.0.0.1:6802/2954588696,v1:127.0.0.1:6803/2954588696] exists,up 9951ed66-ce27-4417-a9c3-080a47cd268e 2026-02-21T11:15:11.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:11.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:11.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:11.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-21T11:15:11.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:11.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:11.201 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:11.201 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.201 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.201 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:11.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:11.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:11.256 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:11.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:11.263 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:07.337+0000 7fedc4278ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-21T11:15:11.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:11.264 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:09.045+0000 7fac0cdbfa40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:11.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:11.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:11.374 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:11.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:11.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:11.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:11.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:11.377 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:11.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:11.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:11.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:11.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:11.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:11.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:11.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:11.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:11.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:11.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:11.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:11.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-21T11:15:11.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:11.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:11.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:11.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:11.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:11.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:11.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:11.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:11.392 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:11.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:11.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:11.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:11.394 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:11.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:11.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:11.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:11.397 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:11.397 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.397 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.397 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:11.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:11.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:11.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:11.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:11.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:11.401 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:11.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:11.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:11.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-21T11:15:11.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:11.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:11.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:11.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:11.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:11.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-21T11:15:11.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:11.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:11.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:11.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:11.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:11.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:11.454 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:11.454 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:11.454 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:11.455 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:11.455 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.455 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.455 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:11.455 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:11.455 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:11.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:11.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:11.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:11.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.512 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:11.513 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:11.513 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:11.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:11.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:11.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:11.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:11.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:11.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:11.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:11.678 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:11.678 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:11.679 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:11.679 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:11.679 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.679 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.679 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:11.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:11.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:11.699 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-21T11:15:11.699 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:11.699 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:11.700 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:11.700 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.700 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.700 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:11.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:11.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:11.751 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:11.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:11.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:11.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:11.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:11.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:11.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-21T11:15:11.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:11.761 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:11.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f0930bcd-c5ca-4e9f-ab44-706a1eac7db2 2026-02-21T11:15:11.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f0930bcd-c5ca-4e9f-ab44-706a1eac7db2' 2026-02-21T11:15:11.762 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 f0930bcd-c5ca-4e9f-ab44-706a1eac7db2 2026-02-21T11:15:11.763 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:11.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC/k5lpK8ETLhAAwUn36dbQIZNUEoMfxvdR7A== 2026-02-21T11:15:11.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC/k5lpK8ETLhAAwUn36dbQIZNUEoMfxvdR7A=="}' 2026-02-21T11:15:11.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f0930bcd-c5ca-4e9f-ab44-706a1eac7db2 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:11.887 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:11.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:11.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQC/k5lpK8ETLhAAwUn36dbQIZNUEoMfxvdR7A== --osd-uuid f0930bcd-c5ca-4e9f-ab44-706a1eac7db2 2026-02-21T11:15:11.911 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:11.909+0000 7f161f656a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:11.913 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:11.909+0000 7f161f656a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:11.913 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:11.909+0000 7f161f656a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:11.914 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:11.913+0000 7f161f656a40 -1 bdev(0x5636d2794400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:11.914 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:11.913+0000 7f161f656a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:12.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:12.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:12.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:12.684 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:12.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:12.796 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:12.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:12.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-21T11:15:12.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:12.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:12.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:12.820 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:12.817+0000 7fc908910a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:12.823 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:12.821+0000 7fc908910a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:12.824 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:12.821+0000 7fc908910a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:12.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:12.904 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:12.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:12.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:13.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:13.180 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:13.177+0000 7fc908910a40 -1 Falling back to public interface 2026-02-21T11:15:14.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:14.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:14.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:14.010 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:14.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:14.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:14.154 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:14.153+0000 7fc908910a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:14.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:15.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:15.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:15.215 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:15.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:15.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:15.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:15.422 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1476270584,v1:127.0.0.1:6801/1476270584] [v2:127.0.0.1:6802/1476270584,v1:127.0.0.1:6803/1476270584] exists,up f0930bcd-c5ca-4e9f-ab44-706a1eac7db2 2026-02-21T11:15:15.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:15.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:15.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:15.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:15.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:15.486 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:15.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:15.493 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:11.437+0000 7fbaf6d23ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-21T11:15:15.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:15.494 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:13.177+0000 7fc908910a40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-21T11:15:15.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:15.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:15.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:15.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:15.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:15.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:15.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:15.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:15.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:15.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:15.604 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:15.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:15.605 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:15.606 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:15.606 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:15.606 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:15.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:15.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:15.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:15.608 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:15.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:15.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:15.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:15.615 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:15.615 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.615 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:15.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:15.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:15.619 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:15.619 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:15.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:15.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:15.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:15.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:15.620 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:15.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:15.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:15.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:15.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:15.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:15.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:15.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:15.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:15.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:15.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:15.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:15.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:15.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:15.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:15.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:15.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:15.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:15.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:15.631 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.631 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:15.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:15.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:15.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:15.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:15.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-21T11:15:15.654 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:15.654 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:15.654 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:15.654 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:15.654 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.654 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:15.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-21T11:15:15.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:15.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:15.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:15.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:15.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:15.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.683 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:15.684 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:15.684 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:15.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:15.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:15.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:15.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:15.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:15.740 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:15.740 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:15.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:15.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:15.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:15.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:15.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:15.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:15.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:15.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:15.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:15.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:15.900 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:15.900 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.900 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.901 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:15.901 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:15.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:15.919 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-21T11:15:15.919 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:15.919 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:15.919 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:15.919 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.919 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.920 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:15.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:15.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:15.973 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:15.984 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:15.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:15.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:15.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-21T11:15:15.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:15.986 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:15.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e123b9ac-1976-49e9-a222-1bb3a242a8b3 2026-02-21T11:15:15.987 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 e123b9ac-1976-49e9-a222-1bb3a242a8b3 2026-02-21T11:15:15.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e123b9ac-1976-49e9-a222-1bb3a242a8b3' 2026-02-21T11:15:15.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:15.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDDk5lpiUFDOxAAA8vIyJaQYlNlxaIj3LHW5w== 2026-02-21T11:15:15.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDDk5lpiUFDOxAAA8vIyJaQYlNlxaIj3LHW5w=="}' 2026-02-21T11:15:15.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e123b9ac-1976-49e9-a222-1bb3a242a8b3 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:16.107 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:16.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:16.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQDDk5lpiUFDOxAAA8vIyJaQYlNlxaIj3LHW5w== --osd-uuid e123b9ac-1976-49e9-a222-1bb3a242a8b3 2026-02-21T11:15:16.132 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:16.129+0000 7f17f068ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:16.133 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:16.129+0000 7f17f068ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:16.134 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:16.133+0000 7f17f068ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:16.135 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:16.133+0000 7f17f068ea40 -1 bdev(0x564d49bee400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:16.135 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:16.133+0000 7f17f068ea40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:16.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:16.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:16.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:16.870 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:16.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:16.984 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:16.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:16.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-21T11:15:16.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:16.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:16.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:17.007 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:17.005+0000 7f092eca0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:17.009 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:17.005+0000 7f092eca0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:17.010 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:17.009+0000 7f092eca0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:17.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:17.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:17.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:17.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:17.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:17.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:17.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:17.094 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:17.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:17.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:17.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:17.415 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:17.409+0000 7f092eca0a40 -1 Falling back to public interface 2026-02-21T11:15:18.018 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:18.017+0000 7f092eca0a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:18.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:18.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:18.198 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:18.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:18.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:18.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:18.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:19.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:19.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:19.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:19.317 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:19.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:19.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:19.519 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1268953965,v1:127.0.0.1:6801/1268953965] [v2:127.0.0.1:6802/1268953965,v1:127.0.0.1:6803/1268953965] exists,up e123b9ac-1976-49e9-a222-1bb3a242a8b3 2026-02-21T11:15:19.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:19.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:19.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:19.582 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:19.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:19.589 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:15.669+0000 7f8edc994ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-21T11:15:19.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:19.590 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:17.409+0000 7f092eca0a40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:19.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:19.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:19.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:19.700 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:19.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:19.701 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:19.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:19.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:19.702 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:19.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:19.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:19.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:19.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:19.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:19.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:19.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:19.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:19.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:19.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:19.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:19.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:19.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:19.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:19.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:19.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:19.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:19.715 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:19.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:19.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:19.717 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:19.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:19.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:19.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:19.718 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:19.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:19.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:19.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:19.721 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:19.721 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.721 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:19.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:19.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:19.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:19.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:19.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:19.726 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:19.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:19.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:19.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:19.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-21T11:15:19.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:19.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:19.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:19.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:19.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:19.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-21T11:15:19.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:19.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:19.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:19.749 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:19.749 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.749 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.749 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:19.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-21T11:15:19.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:19.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:19.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:19.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:19.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:19.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:19.779 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:19.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:19.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:19.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.837 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:19.837 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:19.837 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:19.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:19.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:19.997 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:19.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:20.017 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:20.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:20.068 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:20.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:20.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:20.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:20.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:20.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:20.076 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:20.076 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:20.077 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:20.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b8537cbc-cf66-46bc-9d5c-5447193c6872 2026-02-21T11:15:20.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b8537cbc-cf66-46bc-9d5c-5447193c6872' 2026-02-21T11:15:20.079 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 b8537cbc-cf66-46bc-9d5c-5447193c6872 2026-02-21T11:15:20.079 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:20.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIk5lpXkY+BRAAaJlnA0SiIKJswFwY8WIrMg== 2026-02-21T11:15:20.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIk5lpXkY+BRAAaJlnA0SiIKJswFwY8WIrMg=="}' 2026-02-21T11:15:20.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b8537cbc-cf66-46bc-9d5c-5447193c6872 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:20.199 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:20.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:20.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQDIk5lpXkY+BRAAaJlnA0SiIKJswFwY8WIrMg== --osd-uuid b8537cbc-cf66-46bc-9d5c-5447193c6872 2026-02-21T11:15:20.224 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:20.221+0000 7f27c74a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:20.226 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:20.225+0000 7f27c74a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:20.227 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:20.225+0000 7f27c74a2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:20.227 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:20.225+0000 7f27c74a2a40 -1 bdev(0x559aa791a400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:20.228 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:20.225+0000 7f27c74a2a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:20.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:20.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:20.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:20.894 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:20.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:21.007 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:21.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:21.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-21T11:15:21.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:21.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:21.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:21.022 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:21.021+0000 7f2dd2970a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:21.024 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:21.021+0000 7f2dd2970a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:21.025 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:21.021+0000 7f2dd2970a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:21.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:21.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:21.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:21.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:21.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:21.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:21.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:21.120 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:21.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:21.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:21.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:21.568 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:21.565+0000 7f2dd2970a40 -1 Falling back to public interface 2026-02-21T11:15:22.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:22.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:22.229 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:22.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:22.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:22.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:22.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:22.544 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:22.541+0000 7f2dd2970a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:23.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:23.338 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:23.338 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:23.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:23.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:23.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:23.545 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1206582029,v1:127.0.0.1:6801/1206582029] [v2:127.0.0.1:6802/1206582029,v1:127.0.0.1:6803/1206582029] exists,up b8537cbc-cf66-46bc-9d5c-5447193c6872 2026-02-21T11:15:23.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:23.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:23.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:23.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:23.600 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:23.606 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:23.607 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:19.765+0000 7f3848ac4ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-21T11:15:23.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:23.609 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:21.565+0000 7f2dd2970a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-21T11:15:23.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:23.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:23.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:23.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:23.611 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:23.611 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:23.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:23.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:23.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:23.719 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:23.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:23.720 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:23.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:23.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:23.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:23.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:23.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:23.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:23.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:23.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:23.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:23.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:23.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:23.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:23.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:23.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:23.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:23.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:23.735 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:23.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:23.736 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:23.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:23.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:23.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:23.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:23.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:23.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:23.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:23.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:23.741 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:23.741 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:23.742 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:23.742 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.742 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:23.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:23.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:23.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:23.745 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:23.745 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.745 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:23.746 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:23.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-21T11:15:23.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:23.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:23.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:23.769 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:23.769 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:23.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-21T11:15:23.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:23.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:23.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:23.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:23.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.799 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.800 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:23.800 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:23.800 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:23.858 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:23.859 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:23.860 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:23.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:23.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:23.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:23.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:23.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:23.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:23.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:23.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:23.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:24.024 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:24.025 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:24.025 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:24.025 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:24.025 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:24.025 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:24.025 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:24.026 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:24.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:24.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:24.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:24.094 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:24.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-21T11:15:24.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:24.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:24.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-21T11:15:24.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:24.107 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:24.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c64a7b9e-4a7d-4550-a539-603fd90b3425 2026-02-21T11:15:24.107 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 c64a7b9e-4a7d-4550-a539-603fd90b3425 2026-02-21T11:15:24.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c64a7b9e-4a7d-4550-a539-603fd90b3425' 2026-02-21T11:15:24.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:24.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDMk5lpqi7RBhAADX4v7RXzMZf3Cb+nZfSGqQ== 2026-02-21T11:15:24.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDMk5lpqi7RBhAADX4v7RXzMZf3Cb+nZfSGqQ=="}' 2026-02-21T11:15:24.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c64a7b9e-4a7d-4550-a539-603fd90b3425 -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:24.228 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:24.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:24.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQDMk5lpqi7RBhAADX4v7RXzMZf3Cb+nZfSGqQ== --osd-uuid c64a7b9e-4a7d-4550-a539-603fd90b3425 2026-02-21T11:15:24.253 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:24.249+0000 7f6598c3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:24.254 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:24.253+0000 7f6598c3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:24.255 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:24.253+0000 7f6598c3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:24.256 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:24.253+0000 7f6598c3fa40 -1 bdev(0x556dbf59c400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:24.256 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:24.253+0000 7f6598c3fa40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:25.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:25.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:25.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:25.046 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:25.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:25.166 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-21T11:15:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:25.189 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:25.185+0000 7fdbcad7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:25.190 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:25.189+0000 7fdbcad7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:25.192 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:25.189+0000 7fdbcad7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:25.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:25.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:25.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:25.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:25.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:25.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:25.275 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:25.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:25.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:25.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:25.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:25.604 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:25.601+0000 7fdbcad7fa40 -1 Falling back to public interface 2026-02-21T11:15:26.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:26.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:26.382 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:26.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:26.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:26.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:26.415 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:26.413+0000 7fdbcad7fa40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:26.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:27.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:27.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:27.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:27.594 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:27.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:27.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1078556387,v1:127.0.0.1:6801/1078556387] [v2:127.0.0.1:6802/1078556387,v1:127.0.0.1:6803/1078556387] exists,up c64a7b9e-4a7d-4550-a539-603fd90b3425 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:27.799 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:27.800 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:27.800 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:27.800 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:27.800 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:27.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:27.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:27.862 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:27.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:27.869 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:23.781+0000 7fe2471edec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-21T11:15:27.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:27.870 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:25.601+0000 7fdbcad7fa40 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:27.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:27.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:27.981 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:27.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:27.981 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:27.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:27.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:27.983 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:27.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:27.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:27.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:27.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:27.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:27.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:27.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:27.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:27.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:27.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:27.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:27.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:27.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:27.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:27.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:27.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:27.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:27.995 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:27.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:27.996 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:27.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:27.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:27.998 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:27.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:27.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:27.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:27.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:28.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:28.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:28.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:28.003 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:28.003 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.003 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:28.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:28.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:28.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-21T11:15:28.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:28.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12026 2026-02-21T11:15:28.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:28.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:15:28.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:28.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-21T11:15:28.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-21T11:15:28.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:28.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:28.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:28.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:28.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-21T11:15:28.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:28.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-21T11:15:28.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:28.057 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:28.057 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:28.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:28.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.059 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:28.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:28.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get fsid 2026-02-21T11:15:28.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:28.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:28.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:28.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:28.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:28.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:28.117 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:28.117 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:28.118 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:28.118 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.118 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.118 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:28.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:28.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12026/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-21T11:15:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:28.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:28.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:28.284 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:28.284 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:28.284 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.284 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.284 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:28.284 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:28.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-mon.a.asok 2026-02-21T11:15:28.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:28.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-mon.a.asok log flush 2026-02-21T11:15:28.364 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:28.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:28.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:28.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-21T11:15:28.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-21T11:15:28.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:28.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=31cf342f-1e04-47e2-a999-042ee40cba8c 2026-02-21T11:15:28.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 31cf342f-1e04-47e2-a999-042ee40cba8c' 2026-02-21T11:15:28.379 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 31cf342f-1e04-47e2-a999-042ee40cba8c 2026-02-21T11:15:28.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:28.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDQk5lpMk/7FhAA59H39TKP+nIAbtdGAKSIDg== 2026-02-21T11:15:28.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDQk5lpMk/7FhAA59H39TKP+nIAbtdGAKSIDg=="}' 2026-02-21T11:15:28.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 31cf342f-1e04-47e2-a999-042ee40cba8c -i td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:28.495 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:28.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-21T11:15:28.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQDQk5lpMk/7FhAA59H39TKP+nIAbtdGAKSIDg== --osd-uuid 31cf342f-1e04-47e2-a999-042ee40cba8c 2026-02-21T11:15:28.519 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:28.517+0000 7f3b323b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:28.521 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:28.517+0000 7f3b323b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:28.521 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:28.517+0000 7f3b323b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:28.522 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:28.521+0000 7f3b323b9a40 -1 bdev(0x559071656400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:28.522 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:28.521+0000 7f3b323b9a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-21T11:15:29.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-21T11:15:29.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:29.202 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:29.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:29.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:29.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:29.314 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:29.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c8519c-f270-4105-93bf-5734023ab404 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12026/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-21T11:15:29.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:29.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:29.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:29.328 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:29.325+0000 7f0c573d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:29.329 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:29.325+0000 7f0c573d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:29.330 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:29.329+0000 7f0c573d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:29.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:29.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:29.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:29.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:29.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:29.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:29.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:29.427 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:29.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:29.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:29.530 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:29.719 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:29.717+0000 7f0c573d1a40 -1 Falling back to public interface 2026-02-21T11:15:30.419 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:30.417+0000 7f0c573d1a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:30.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:30.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:30.532 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:30.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:30.533 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:30.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:30.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:31.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:31.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:31.661 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:31.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:31.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:31.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:31.874 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1034984942,v1:127.0.0.1:6801/1034984942] [v2:127.0.0.1:6802/1034984942,v1:127.0.0.1:6803/1034984942] exists,up 31cf342f-1e04-47e2-a999-042ee40cba8c 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:15:31.876 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:31.877 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:31.877 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:31.877 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12026/ceph-osd.0.asok 2026-02-21T11:15:31.877 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-21T11:15:31.877 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12026/ceph-osd.0.asok log flush 2026-02-21T11:15:31.939 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:31.944 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-21T11:15:31.946 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:28.045+0000 7f8307d98ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-21T11:15:31.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-21T11:15:31.947 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:29.717+0000 7f0c573d1a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-21T11:15:31.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:31.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:32.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:32.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:32.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:32.059 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:32.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:32.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:32.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:32.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:32.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:32.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:32.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:32.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:32.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:32.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:32.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:32.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:32.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/test-erasure-code-plugins 0 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T11:15:32.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-21T11:15:32.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:32.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:32.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:32.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:32.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:32.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:32.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:32.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:32.073 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:32.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:32.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:32.075 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:32.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:32.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:32.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:32.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:32.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:32.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T11:15:32.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-21T11:15:32.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:32.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12026 2026-02-21T11:15:32.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12026 2026-02-21T11:15:32.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:32.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:32.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-21T11:15:32.083 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T11:15:32.084 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T11:15:32.093 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-21T11:15:32.094 DEBUG:teuthology.orchestra.run.trial166:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7273f3049146bdd7fa7b4a400114872a50228e9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-21T11:15:32.143 INFO:tasks.workunit.client.0.trial166.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T11:15:32.147 INFO:tasks.workunit.client.0.trial166.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T11:15:32.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T11:15:32.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T11:15:32.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-21T11:15:32.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-21T11:15:32.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/test-erasure-code 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-21T11:15:32.148 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none ' 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:15:32.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-21T11:15:32.151 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:15:32.151 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:15:32.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:15:32.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:15:32.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:15:32.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:15:32.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:15:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:15:32.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:15:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:15:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:15:32.153 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:15:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:15:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:15:32.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:15:32.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:15:32.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:15:32.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-21T11:15:32.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:15:32.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46206 2026-02-21T11:15:32.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:15:32.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:15:32.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-21T11:15:32.160 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:15:32.161 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.161 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.46206 2026-02-21T11:15:32.162 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:15:32.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-21T11:15:32.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-21T11:15:32.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:15:32.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:32.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:32.188 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:32.188 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.188 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.189 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:32.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:15:32.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:15:32.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:15:32.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:32.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:32.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:15:32.218 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:15:32.218 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:32.218 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:32.218 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:32.219 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:32.219 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.219 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.219 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46206/ceph-mon.a.asok 2026-02-21T11:15:32.219 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:32.219 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46206/ceph-mon.a.asok config get fsid 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:15:32.275 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:32.276 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:32.276 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:32.276 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.277 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.277 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46206/ceph-mon.a.asok 2026-02-21T11:15:32.277 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:15:32.277 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46206/ceph-mon.a.asok config get mon_host 2026-02-21T11:15:32.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-21T11:15:32.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-21T11:15:32.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:15:32.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:15:32.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:15:32.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-21T11:15:32.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:15:32.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:15:32.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:32.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:32.440 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:32.440 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.440 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:32.441 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:32.441 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46206/ceph-mon.a.asok 2026-02-21T11:15:32.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-21T11:15:32.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.46206/ceph-mon.a.asok log flush 2026-02-21T11:15:32.512 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:15:32.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-21T11:15:32.519 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:32.205+0000 7f9fba463ec0 0 load: jerasure load: lrc 2026-02-21T11:15:32.520 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-21T11:15:32.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:32.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-21T11:15:32.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:32.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:32.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:32.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:32.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-21T11:15:32.527 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:32.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7189c99f-04e5-45c5-8038-a00eed6ebb1f 2026-02-21T11:15:32.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7189c99f-04e5-45c5-8038-a00eed6ebb1f' 2026-02-21T11:15:32.527 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 7189c99f-04e5-45c5-8038-a00eed6ebb1f 2026-02-21T11:15:32.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:32.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDUk5lp5UfUHxAA08qYNBtCw0Ys5v/2eAyzsw== 2026-02-21T11:15:32.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDUk5lp5UfUHxAA08qYNBtCw0Ys5v/2eAyzsw=="}' 2026-02-21T11:15:32.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7189c99f-04e5-45c5-8038-a00eed6ebb1f -i td/test-erasure-code/0/new.json 2026-02-21T11:15:32.647 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:32.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-21T11:15:32.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUk5lp5UfUHxAA08qYNBtCw0Ys5v/2eAyzsw== --osd-uuid 7189c99f-04e5-45c5-8038-a00eed6ebb1f 2026-02-21T11:15:32.673 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:32.669+0000 7f80474d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:32.674 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:32.673+0000 7f80474d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:32.676 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:32.673+0000 7f80474d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:32.676 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:32.673+0000 7f80474d8a40 -1 bdev(0x56312d1c4400 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:32.676 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:32.673+0000 7f80474d8a40 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-21T11:15:33.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-21T11:15:33.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:33.432 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:15:33.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:15:33.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:33.555 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:15:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:15:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:33.579 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:33.577+0000 7f59ab310a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:33.580 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:33.577+0000 7f59ab310a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:33.582 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:33.581+0000 7f59ab310a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:33.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:15:33.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:33.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:15:33.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:33.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:33.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:33.666 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:33.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:33.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:33.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:33.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:34.012 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:34.009+0000 7f59ab310a40 -1 Falling back to public interface 2026-02-21T11:15:34.691 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:34.689+0000 7f59ab310a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:15:34.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:34.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:34.781 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:34.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:34.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:34.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:34.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:35.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:35.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:35.997 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:35.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:35.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:35.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:15:36.206 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1469561931,v1:127.0.0.1:6801/1469561931] [v2:127.0.0.1:6802/1469561931,v1:127.0.0.1:6803/1469561931] exists,up 7189c99f-04e5-45c5-8038-a00eed6ebb1f 2026-02-21T11:15:36.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:36.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:36.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:36.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:36.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-21T11:15:36.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:36.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6432f99b-d40e-41f4-82b4-c007c1526915 2026-02-21T11:15:36.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6432f99b-d40e-41f4-82b4-c007c1526915' 2026-02-21T11:15:36.214 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 6432f99b-d40e-41f4-82b4-c007c1526915 2026-02-21T11:15:36.214 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:36.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYk5lphotCDRAACQCHdrkpAlbhXdcR83dgeQ== 2026-02-21T11:15:36.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYk5lphotCDRAACQCHdrkpAlbhXdcR83dgeQ=="}' 2026-02-21T11:15:36.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6432f99b-d40e-41f4-82b4-c007c1526915 -i td/test-erasure-code/1/new.json 2026-02-21T11:15:36.436 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-21T11:15:36.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDYk5lphotCDRAACQCHdrkpAlbhXdcR83dgeQ== --osd-uuid 6432f99b-d40e-41f4-82b4-c007c1526915 2026-02-21T11:15:36.462 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:36.461+0000 7fcb84780a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:36.463 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:36.461+0000 7fcb84780a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:36.464 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:36.461+0000 7fcb84780a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:36.464 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:36.461+0000 7fcb84780a40 -1 bdev(0x5652b584a400 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:36.465 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:36.461+0000 7fcb84780a40 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-21T11:15:37.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-21T11:15:37.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:37.098 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:15:37.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:15:37.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:37.393 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:15:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:15:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:37.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:37.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:37.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:37.416 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:37.413+0000 7f5fad372a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:37.418 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:37.417+0000 7f5fad372a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:37.419 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:37.417+0000 7f5fad372a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:37.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:15:37.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:37.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:15:37.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:37.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:37.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:37.604 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:37.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:37.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:37.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:15:37.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:37.848 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:37.845+0000 7f5fad372a40 -1 Falling back to public interface 2026-02-21T11:15:38.464 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:38.461+0000 7f5fad372a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:15:38.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:38.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:38.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:38.818 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:38.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:38.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:15:39.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:40.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:40.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:40.026 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:40.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:40.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:40.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:15:40.229 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3681232007,v1:127.0.0.1:6811/3681232007] [v2:127.0.0.1:6812/3681232007,v1:127.0.0.1:6813/3681232007] exists,up 6432f99b-d40e-41f4-82b4-c007c1526915 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:40.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:40.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:40.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-21T11:15:40.233 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:40.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=54a86d68-cce6-4cbc-ba3f-0c00f5aa4197 2026-02-21T11:15:40.233 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 54a86d68-cce6-4cbc-ba3f-0c00f5aa4197 2026-02-21T11:15:40.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 54a86d68-cce6-4cbc-ba3f-0c00f5aa4197' 2026-02-21T11:15:40.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:40.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDck5lpaqmqDhAAOSeYaWEwq9Z8KZx1+866aA== 2026-02-21T11:15:40.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDck5lpaqmqDhAAOSeYaWEwq9Z8KZx1+866aA=="}' 2026-02-21T11:15:40.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 54a86d68-cce6-4cbc-ba3f-0c00f5aa4197 -i td/test-erasure-code/2/new.json 2026-02-21T11:15:40.465 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:40.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-21T11:15:40.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDck5lpaqmqDhAAOSeYaWEwq9Z8KZx1+866aA== --osd-uuid 54a86d68-cce6-4cbc-ba3f-0c00f5aa4197 2026-02-21T11:15:40.500 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:40.496+0000 7f5018265a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:40.501 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:40.496+0000 7f5018265a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:40.502 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:40.500+0000 7f5018265a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:40.502 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:40.500+0000 7f5018265a40 -1 bdev(0x55d8bf574400 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:40.503 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:40.500+0000 7f5018265a40 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-21T11:15:41.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-21T11:15:41.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:41.142 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:15:41.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:15:41.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:41.426 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:15:41.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:15:41.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:41.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:41.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:41.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:41.451 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:41.448+0000 7f3a62725a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:41.453 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:41.448+0000 7f3a62725a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:41.454 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:41.452+0000 7f3a62725a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:41.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:15:41.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:41.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:15:41.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:41.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:41.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:41.637 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:41.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:41.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:41.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:15:41.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:41.895 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:41.892+0000 7f3a62725a40 -1 Falling back to public interface 2026-02-21T11:15:42.568 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:42.564+0000 7f3a62725a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:15:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:42.850 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:15:43.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:44.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:44.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:44.064 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:44.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:44.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:44.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:15:44.274 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3556879136,v1:127.0.0.1:6819/3556879136] [v2:127.0.0.1:6820/3556879136,v1:127.0.0.1:6821/3556879136] exists,up 54a86d68-cce6-4cbc-ba3f-0c00f5aa4197 2026-02-21T11:15:44.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:44.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:44.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:44.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:44.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-21T11:15:44.280 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:44.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b5515ea-f993-49ac-a875-4b34cfde9205 2026-02-21T11:15:44.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 1b5515ea-f993-49ac-a875-4b34cfde9205' 2026-02-21T11:15:44.282 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 1b5515ea-f993-49ac-a875-4b34cfde9205 2026-02-21T11:15:44.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:44.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgk5lpaZVMERAAjshL5GfE6AAC3k/QaIyQ8A== 2026-02-21T11:15:44.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgk5lpaZVMERAAjshL5GfE6AAC3k/QaIyQ8A=="}' 2026-02-21T11:15:44.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b5515ea-f993-49ac-a875-4b34cfde9205 -i td/test-erasure-code/3/new.json 2026-02-21T11:15:44.512 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:15:44.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-21T11:15:44.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgk5lpaZVMERAAjshL5GfE6AAC3k/QaIyQ8A== --osd-uuid 1b5515ea-f993-49ac-a875-4b34cfde9205 2026-02-21T11:15:44.548 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:44.544+0000 7f9c87384a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:44.549 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:44.544+0000 7f9c87384a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:44.550 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:44.548+0000 7f9c87384a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:44.551 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:44.548+0000 7f9c87384a40 -1 bdev(0x559fa4022400 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:44.551 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:44.548+0000 7f9c87384a40 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-21T11:15:45.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-21T11:15:45.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:45.214 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:15:45.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:15:45.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:45.499 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:15:45.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:15:45.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:45.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:45.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:45.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:45.522 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:45.516+0000 7f3adb08ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:45.524 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:45.520+0000 7f3adb08ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:45.525 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:45.520+0000 7f3adb08ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:45.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:15:45.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:45.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:15:45.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:45.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:45.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:45.714 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:45.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:45.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:45.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:15:45.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:45.975 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:45.972+0000 7f3adb08ea40 -1 Falling back to public interface 2026-02-21T11:15:46.563 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:46.560+0000 7f3adb08ea40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:15:46.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:46.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:46.925 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:46.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:46.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:46.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:15:47.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:48.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:48.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:48.133 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:48.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:48.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:48.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:15:48.341 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1707270541,v1:127.0.0.1:6827/1707270541] [v2:127.0.0.1:6828/1707270541,v1:127.0.0.1:6829/1707270541] exists,up 1b5515ea-f993-49ac-a875-4b34cfde9205 2026-02-21T11:15:48.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:48.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:48.346 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:48.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:48.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:48.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:48.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:48.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-21T11:15:48.348 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:48.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cf3aa453-5eab-424f-827d-f1a7978774c5 2026-02-21T11:15:48.349 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 cf3aa453-5eab-424f-827d-f1a7978774c5 2026-02-21T11:15:48.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 cf3aa453-5eab-424f-827d-f1a7978774c5' 2026-02-21T11:15:48.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:48.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDkk5lprNtSFRAAloN1RdfdgWFPvJmtBhIO6w== 2026-02-21T11:15:48.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDkk5lprNtSFRAAloN1RdfdgWFPvJmtBhIO6w=="}' 2026-02-21T11:15:48.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cf3aa453-5eab-424f-827d-f1a7978774c5 -i td/test-erasure-code/4/new.json 2026-02-21T11:15:48.573 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:15:48.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-21T11:15:48.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDkk5lprNtSFRAAloN1RdfdgWFPvJmtBhIO6w== --osd-uuid cf3aa453-5eab-424f-827d-f1a7978774c5 2026-02-21T11:15:48.607 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:48.604+0000 7efd5ca1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:48.609 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:48.604+0000 7efd5ca1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:48.611 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:48.608+0000 7efd5ca1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:48.611 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:48.608+0000 7efd5ca1ba40 -1 bdev(0x560aa3154400 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:48.611 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:48.608+0000 7efd5ca1ba40 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-21T11:15:49.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-21T11:15:49.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:49.245 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:15:49.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:15:49.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:49.530 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:15:49.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:15:49.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:49.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:49.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:49.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:49.553 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:49.548+0000 7f916905ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:49.555 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:49.552+0000 7f916905ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:49.556 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:49.552+0000 7f916905ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:49.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:15:49.953 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:49.948+0000 7f916905ba40 -1 Falling back to public interface 2026-02-21T11:15:49.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:50.557 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:50.552+0000 7f916905ba40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:15:50.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:50.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:50.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:50.962 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:50.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:50.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:15:51.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:52.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:52.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:52.170 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:52.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:52.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:52.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:15:52.377 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1349781383,v1:127.0.0.1:6835/1349781383] [v2:127.0.0.1:6836/1349781383,v1:127.0.0.1:6837/1349781383] exists,up cf3aa453-5eab-424f-827d-f1a7978774c5 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:52.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:52.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:52.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-21T11:15:52.381 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:52.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cc03670a-b315-45a4-bd4f-afa125bcdeb6 2026-02-21T11:15:52.381 INFO:tasks.workunit.client.0.trial166.stdout:add osd5 cc03670a-b315-45a4-bd4f-afa125bcdeb6 2026-02-21T11:15:52.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 cc03670a-b315-45a4-bd4f-afa125bcdeb6' 2026-02-21T11:15:52.382 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:52.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDok5lpT2RkFxAAhl26NkZOm4E/5kXJ2uJZYw== 2026-02-21T11:15:52.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDok5lpT2RkFxAAhl26NkZOm4E/5kXJ2uJZYw=="}' 2026-02-21T11:15:52.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cc03670a-b315-45a4-bd4f-afa125bcdeb6 -i td/test-erasure-code/5/new.json 2026-02-21T11:15:52.617 INFO:tasks.workunit.client.0.trial166.stdout:5 2026-02-21T11:15:52.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-21T11:15:52.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDok5lpT2RkFxAAhl26NkZOm4E/5kXJ2uJZYw== --osd-uuid cc03670a-b315-45a4-bd4f-afa125bcdeb6 2026-02-21T11:15:52.651 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:52.648+0000 7f83b6936a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:52.653 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:52.648+0000 7f83b6936a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:52.654 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:52.648+0000 7f83b6936a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:52.654 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:52.652+0000 7f83b6936a40 -1 bdev(0x55946baba400 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:52.654 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:52.652+0000 7f83b6936a40 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-21T11:15:53.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-21T11:15:53.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:53.234 INFO:tasks.workunit.client.0.trial166.stdout:adding osd5 key to auth repository 2026-02-21T11:15:53.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-21T11:15:53.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:53.531 INFO:tasks.workunit.client.0.trial166.stdout:start osd.5 2026-02-21T11:15:53.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-21T11:15:53.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:53.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:53.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:53.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:53.546 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:53.544+0000 7fec2c29da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:53.548 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:53.544+0000 7fec2c29da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:53.549 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:53.544+0000 7fec2c29da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:53.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-21T11:15:53.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:53.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:15:53.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:53.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:53.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:53.744 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:53.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:53.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:53.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:15:53.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:54.039 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:54.036+0000 7fec2c29da40 -1 Falling back to public interface 2026-02-21T11:15:54.629 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:54.624+0000 7fec2c29da40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:15:54.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:54.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:54.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:54.955 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:54.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:54.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:15:55.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:56.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:56.175 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:15:56.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:56.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:15:56.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:56.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:15:56.387 INFO:tasks.workunit.client.0.trial166.stdout:osd.5 up in weight 1 up_from 29 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1611717158,v1:127.0.0.1:6843/1611717158] [v2:127.0.0.1:6844/1611717158,v1:127.0.0.1:6845/1611717158] exists,up cc03670a-b315-45a4-bd4f-afa125bcdeb6 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:56.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:56.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:56.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-21T11:15:56.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:56.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1810de0f-541f-419e-9ac7-adfe593930ea 2026-02-21T11:15:56.392 INFO:tasks.workunit.client.0.trial166.stdout:add osd6 1810de0f-541f-419e-9ac7-adfe593930ea 2026-02-21T11:15:56.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 1810de0f-541f-419e-9ac7-adfe593930ea' 2026-02-21T11:15:56.392 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:56.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDsk5lpcugGGBAAZ9Ip7KCnI33+OY41dtOAaQ== 2026-02-21T11:15:56.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDsk5lpcugGGBAAZ9Ip7KCnI33+OY41dtOAaQ=="}' 2026-02-21T11:15:56.404 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1810de0f-541f-419e-9ac7-adfe593930ea -i td/test-erasure-code/6/new.json 2026-02-21T11:15:56.623 INFO:tasks.workunit.client.0.trial166.stdout:6 2026-02-21T11:15:56.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-21T11:15:56.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDsk5lpcugGGBAAZ9Ip7KCnI33+OY41dtOAaQ== --osd-uuid 1810de0f-541f-419e-9ac7-adfe593930ea 2026-02-21T11:15:56.658 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:56.656+0000 7f8b7563ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:56.660 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:56.656+0000 7f8b7563ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:56.661 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:56.656+0000 7f8b7563ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:56.661 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:56.656+0000 7f8b7563ca40 -1 bdev(0x55646454a400 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:56.661 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:56.656+0000 7f8b7563ca40 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-21T11:15:57.257 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-21T11:15:57.257 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:15:57.258 INFO:tasks.workunit.client.0.trial166.stdout:adding osd6 key to auth repository 2026-02-21T11:15:57.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-21T11:15:57.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:15:57.558 INFO:tasks.workunit.client.0.trial166.stdout:start osd.6 2026-02-21T11:15:57.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-21T11:15:57.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:15:57.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:15:57.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:15:57.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:15:57.574 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:57.572+0000 7fa4f9367a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:57.576 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:57.572+0000 7fa4f9367a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:57.577 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:57.572+0000 7fa4f9367a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:57.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-21T11:15:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:15:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:15:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:15:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:15:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:15:57.772 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:15:57.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:57.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:15:57.947 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:57.944+0000 7fa4f9367a40 -1 Falling back to public interface 2026-02-21T11:15:57.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:15:58.507 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:58.504+0000 7fa4f9367a40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:15:58.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:15:58.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:15:58.990 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:15:58.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:15:58.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:15:58.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:15:59.197 INFO:tasks.workunit.client.0.trial166.stdout:osd.6 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/484890368,v1:127.0.0.1:6851/484890368] [v2:127.0.0.1:6852/484890368,v1:127.0.0.1:6853/484890368] exists,up 1810de0f-541f-419e-9ac7-adfe593930ea 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-21T11:15:59.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:15:59.202 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:15:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:15:59.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:15:59.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:15:59.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:15:59.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:15:59.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-21T11:15:59.204 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:15:59.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8c2daa41-ebd9-444f-9e8a-bf0fb61205aa 2026-02-21T11:15:59.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 8c2daa41-ebd9-444f-9e8a-bf0fb61205aa' 2026-02-21T11:15:59.205 INFO:tasks.workunit.client.0.trial166.stdout:add osd7 8c2daa41-ebd9-444f-9e8a-bf0fb61205aa 2026-02-21T11:15:59.205 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:15:59.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDvk5lpzNSoDBAA1AvWx+XlJP0j5ifk/pb1kg== 2026-02-21T11:15:59.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDvk5lpzNSoDBAA1AvWx+XlJP0j5ifk/pb1kg=="}' 2026-02-21T11:15:59.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8c2daa41-ebd9-444f-9e8a-bf0fb61205aa -i td/test-erasure-code/7/new.json 2026-02-21T11:15:59.435 INFO:tasks.workunit.client.0.trial166.stdout:7 2026-02-21T11:15:59.444 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-21T11:15:59.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDvk5lpzNSoDBAA1AvWx+XlJP0j5ifk/pb1kg== --osd-uuid 8c2daa41-ebd9-444f-9e8a-bf0fb61205aa 2026-02-21T11:15:59.460 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:59.456+0000 7f21fe50fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:59.462 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:59.456+0000 7f21fe50fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:59.463 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:59.460+0000 7f21fe50fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:15:59.463 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:59.460+0000 7f21fe50fa40 -1 bdev(0x560b2f6f4400 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-21T11:15:59.464 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:15:59.460+0000 7f21fe50fa40 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-21T11:16:00.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-21T11:16:00.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:16:00.162 INFO:tasks.workunit.client.0.trial166.stdout:adding osd7 key to auth repository 2026-02-21T11:16:00.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-21T11:16:00.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:16:00.437 INFO:tasks.workunit.client.0.trial166.stdout:start osd.7 2026-02-21T11:16:00.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-21T11:16:00.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:00.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:16:00.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:16:00.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:16:00.462 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:00.456+0000 7f1539850a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:00.464 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:00.460+0000 7f1539850a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:00.465 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:00.460+0000 7f1539850a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:00.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-21T11:16:00.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:00.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-21T11:16:00.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:00.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:00.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:00.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:00.643 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:16:00.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:00.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-21T11:16:00.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:00.900 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:00.896+0000 7f1539850a40 -1 Falling back to public interface 2026-02-21T11:16:01.507 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:01.504+0000 7f1539850a40 -1 osd.7 0 log_to_monitors true 2026-02-21T11:16:01.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:01.852 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:16:01.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:01.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:01.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:01.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-21T11:16:02.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:03.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:03.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:03.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:16:03.074 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:16:03.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:03.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-21T11:16:03.281 INFO:tasks.workunit.client.0.trial166.stdout:osd.7 up in weight 1 up_from 37 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/3767916176,v1:127.0.0.1:6859/3767916176] [v2:127.0.0.1:6860/3767916176,v1:127.0.0.1:6861/3767916176] exists,up 8c2daa41-ebd9-444f-9e8a-bf0fb61205aa 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:03.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:03.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:03.285 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:03.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a7999de0-c15e-43f3-8b04-95de02f25878 2026-02-21T11:16:03.287 INFO:tasks.workunit.client.0.trial166.stdout:add osd8 a7999de0-c15e-43f3-8b04-95de02f25878 2026-02-21T11:16:03.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 a7999de0-c15e-43f3-8b04-95de02f25878' 2026-02-21T11:16:03.288 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:16:03.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDzk5lplSevERAAA69rGkxH6La0PrLHxtDDUQ== 2026-02-21T11:16:03.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDzk5lplSevERAAA69rGkxH6La0PrLHxtDDUQ=="}' 2026-02-21T11:16:03.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a7999de0-c15e-43f3-8b04-95de02f25878 -i td/test-erasure-code/8/new.json 2026-02-21T11:16:03.509 INFO:tasks.workunit.client.0.trial166.stdout:8 2026-02-21T11:16:03.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-21T11:16:03.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDzk5lplSevERAAA69rGkxH6La0PrLHxtDDUQ== --osd-uuid a7999de0-c15e-43f3-8b04-95de02f25878 2026-02-21T11:16:03.535 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:03.532+0000 7fa41a50da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:03.536 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:03.532+0000 7fa41a50da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:03.537 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:03.532+0000 7fa41a50da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:03.538 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:03.536+0000 7fa41a50da40 -1 bdev(0x562101b38400 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-21T11:16:03.538 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:03.536+0000 7fa41a50da40 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-21T11:16:04.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-21T11:16:04.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:16:04.194 INFO:tasks.workunit.client.0.trial166.stdout:adding osd8 key to auth repository 2026-02-21T11:16:04.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-21T11:16:04.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:16:04.471 INFO:tasks.workunit.client.0.trial166.stdout:start osd.8 2026-02-21T11:16:04.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-21T11:16:04.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:04.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:16:04.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:16:04.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:16:04.494 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:04.492+0000 7fe47975da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:04.496 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:04.492+0000 7fe47975da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:04.497 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:04.492+0000 7fe47975da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:04.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-21T11:16:04.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:04.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-21T11:16:04.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:04.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:04.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:04.688 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:16:04.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:04.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:04.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-21T11:16:04.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:04.947 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:04.944+0000 7fe47975da40 -1 Falling back to public interface 2026-02-21T11:16:05.535 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:05.532+0000 7fe47975da40 -1 osd.8 0 log_to_monitors true 2026-02-21T11:16:05.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:05.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:05.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:05.901 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:16:05.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:05.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-21T11:16:06.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:07.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:07.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:07.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:16:07.113 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:16:07.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:07.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-21T11:16:07.314 INFO:tasks.workunit.client.0.trial166.stdout:osd.8 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/3671887568,v1:127.0.0.1:6867/3671887568] [v2:127.0.0.1:6868/3671887568,v1:127.0.0.1:6869/3671887568] exists,up a7999de0-c15e-43f3-8b04-95de02f25878 2026-02-21T11:16:07.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:07.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:07.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:07.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:16:07.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-21T11:16:07.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:16:07.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:16:07.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-21T11:16:07.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:16:07.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-21T11:16:07.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:07.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:07.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:07.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:07.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-21T11:16:07.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-21T11:16:07.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:07.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:16:07.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:07.322 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:16:07.322 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:07.322 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:07.322 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:07.322 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:07.322 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:07.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:07.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:07.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:07.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:07.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:07.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:16:07.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:16:07.325 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-21T11:16:07.325 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:16:07.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=af42a358-deac-40d5-ba47-33a4947c7d7d 2026-02-21T11:16:07.326 INFO:tasks.workunit.client.0.trial166.stdout:add osd9 af42a358-deac-40d5-ba47-33a4947c7d7d 2026-02-21T11:16:07.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 af42a358-deac-40d5-ba47-33a4947c7d7d' 2026-02-21T11:16:07.327 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:16:07.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3k5lpOpPCExAAUPcCCAL6zZieKsHaCainPQ== 2026-02-21T11:16:07.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3k5lpOpPCExAAUPcCCAL6zZieKsHaCainPQ=="}' 2026-02-21T11:16:07.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new af42a358-deac-40d5-ba47-33a4947c7d7d -i td/test-erasure-code/9/new.json 2026-02-21T11:16:07.553 INFO:tasks.workunit.client.0.trial166.stdout:9 2026-02-21T11:16:07.563 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-21T11:16:07.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3k5lpOpPCExAAUPcCCAL6zZieKsHaCainPQ== --osd-uuid af42a358-deac-40d5-ba47-33a4947c7d7d 2026-02-21T11:16:07.581 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:07.576+0000 7f6f2be8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:07.582 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:07.580+0000 7f6f2be8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:07.583 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:07.580+0000 7f6f2be8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:07.584 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:07.580+0000 7f6f2be8ca40 -1 bdev(0x55e99c6c6400 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-21T11:16:07.584 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:07.580+0000 7f6f2be8ca40 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-21T11:16:08.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-21T11:16:08.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:16:08.270 INFO:tasks.workunit.client.0.trial166.stdout:adding osd9 key to auth repository 2026-02-21T11:16:08.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-21T11:16:08.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:16:08.554 INFO:tasks.workunit.client.0.trial166.stdout:start osd.9 2026-02-21T11:16:08.555 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-21T11:16:08.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:08.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:16:08.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:16:08.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:16:08.576 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:08.572+0000 7fabd7699a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:08.578 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:08.576+0000 7fabd7699a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:08.579 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:08.576+0000 7fabd7699a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:08.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-21T11:16:08.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:08.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-21T11:16:08.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:08.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:08.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:08.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:08.773 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:16:08.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:08.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-21T11:16:08.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:08.987 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:08.984+0000 7fabd7699a40 -1 Falling back to public interface 2026-02-21T11:16:09.585 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:09.580+0000 7fabd7699a40 -1 osd.9 0 log_to_monitors true 2026-02-21T11:16:09.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:09.987 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:16:09.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:09.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:09.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:09.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-21T11:16:10.195 INFO:tasks.workunit.client.0.trial166.stdout:osd.9 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/958440244,v1:127.0.0.1:6875/958440244] [v2:127.0.0.1:6876/958440244,v1:127.0.0.1:6877/958440244] exists,up af42a358-deac-40d5-ba47-33a4947c7d7d 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:16:10.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:10.198 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:10.199 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:10.199 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:10.199 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:10.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:10.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:10.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:16:10.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cd7a98ce-35dd-4506-b66c-f53b34b8ba72 2026-02-21T11:16:10.201 INFO:tasks.workunit.client.0.trial166.stdout:add osd10 cd7a98ce-35dd-4506-b66c-f53b34b8ba72 2026-02-21T11:16:10.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 cd7a98ce-35dd-4506-b66c-f53b34b8ba72' 2026-02-21T11:16:10.202 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:16:10.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6k5lpM5ulDBAA5jBS5svh29FP/70yLzhI8w== 2026-02-21T11:16:10.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6k5lpM5ulDBAA5jBS5svh29FP/70yLzhI8w=="}' 2026-02-21T11:16:10.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cd7a98ce-35dd-4506-b66c-f53b34b8ba72 -i td/test-erasure-code/10/new.json 2026-02-21T11:16:10.435 INFO:tasks.workunit.client.0.trial166.stdout:10 2026-02-21T11:16:10.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-21T11:16:10.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6k5lpM5ulDBAA5jBS5svh29FP/70yLzhI8w== --osd-uuid cd7a98ce-35dd-4506-b66c-f53b34b8ba72 2026-02-21T11:16:10.468 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:10.464+0000 7fc337cc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:10.470 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:10.468+0000 7fc337cc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:10.471 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:10.468+0000 7fc337cc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:10.472 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:10.468+0000 7fc337cc1a40 -1 bdev(0x563658528400 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-21T11:16:10.472 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:10.468+0000 7fc337cc1a40 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-21T11:16:11.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-21T11:16:11.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:16:11.186 INFO:tasks.workunit.client.0.trial166.stdout:adding osd10 key to auth repository 2026-02-21T11:16:11.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-21T11:16:11.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:16:11.474 INFO:tasks.workunit.client.0.trial166.stdout:start osd.10 2026-02-21T11:16:11.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-21T11:16:11.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:11.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:16:11.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:16:11.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:16:11.498 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:11.496+0000 7f6d9cc21a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:11.499 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:11.496+0000 7f6d9cc21a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:11.500 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:11.496+0000 7f6d9cc21a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:11.685 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-21T11:16:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-21T11:16:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:11.687 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:16:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-21T11:16:11.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:11.972 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:11.968+0000 7f6d9cc21a40 -1 Falling back to public interface 2026-02-21T11:16:12.580 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:12.576+0000 7f6d9cc21a40 -1 osd.10 0 log_to_monitors true 2026-02-21T11:16:12.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:12.895 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:16:12.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:12.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:12.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:12.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-21T11:16:13.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:14.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:14.113 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:16:14.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:14.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:16:14.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:14.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-21T11:16:14.319 INFO:tasks.workunit.client.0.trial166.stdout:osd.10 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/822949204,v1:127.0.0.1:6883/822949204] [v2:127.0.0.1:6884/822949204,v1:127.0.0.1:6885/822949204] exists,up cd7a98ce-35dd-4506-b66c-f53b34b8ba72 2026-02-21T11:16:14.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:14.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:14.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:14.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-21T11:16:14.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-21T11:16:14.523 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' does not exist 2026-02-21T11:16:14.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-21T11:16:14.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:16:14.766 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:16:14.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:16:15.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-21T11:16:16.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-21T11:16:16.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:16:16.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:16:16.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:16:16.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:16:16.160 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:16:16.160 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:16:16.160 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:16:16.160 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:16:16.160 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:16:16.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:16:16.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:16:16.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:16:16.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:16:16.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:16:16.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:16:16.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:16:16.467 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:16.467 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:16:16.467 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:16:16.467 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:16:16.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836491 2026-02-21T11:16:16.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836491 2026-02-21T11:16:16.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491' 2026-02-21T11:16:16.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:16:16.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672970 2026-02-21T11:16:16.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672970 2026-02-21T11:16:16.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970' 2026-02-21T11:16:16.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:16:16.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509449 2026-02-21T11:16:16.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509449 2026-02-21T11:16:16.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449' 2026-02-21T11:16:16.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.702 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:16:16.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=85899345928 2026-02-21T11:16:16.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 85899345928 2026-02-21T11:16:16.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928' 2026-02-21T11:16:16.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:16:16.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182408 2026-02-21T11:16:16.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182408 2026-02-21T11:16:16.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408' 2026-02-21T11:16:16.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:16:16.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=124554051591 2026-02-21T11:16:16.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 124554051591 2026-02-21T11:16:16.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408 5-124554051591' 2026-02-21T11:16:16.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:16.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:16:17.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920774 2026-02-21T11:16:17.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920774 2026-02-21T11:16:17.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408 5-124554051591 6-141733920774' 2026-02-21T11:16:17.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:17.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:16:17.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789958 2026-02-21T11:16:17.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789958 2026-02-21T11:16:17.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408 5-124554051591 6-141733920774 7-158913789958' 2026-02-21T11:16:17.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:17.100 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:16:17.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626437 2026-02-21T11:16:17.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626437 2026-02-21T11:16:17.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408 5-124554051591 6-141733920774 7-158913789958 8-180388626437' 2026-02-21T11:16:17.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:17.180 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:16:17.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=201863462917 2026-02-21T11:16:17.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 201863462917 2026-02-21T11:16:17.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408 5-124554051591 6-141733920774 7-158913789958 8-180388626437 9-201863462917' 2026-02-21T11:16:17.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:17.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:16:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332099 2026-02-21T11:16:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332099 2026-02-21T11:16:17.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449 3-85899345928 4-107374182408 5-124554051591 6-141733920774 7-158913789958 8-180388626437 9-201863462917 10-219043332099' 2026-02-21T11:16:17.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:17.341 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836491 2026-02-21T11:16:17.341 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:17.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:16:17.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836491 2026-02-21T11:16:17.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:17.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836491 2026-02-21T11:16:17.344 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836491 2026-02-21T11:16:17.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-21T11:16:17.345 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:17.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-21T11:16:17.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:17.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672970 2026-02-21T11:16:17.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:17.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:16:17.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672970 2026-02-21T11:16:17.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:17.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672970 2026-02-21T11:16:17.566 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 42949672970 2026-02-21T11:16:17.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-21T11:16:17.567 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:16:17.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-21T11:16:17.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:17.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509449 2026-02-21T11:16:17.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:17.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:16:17.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509449 2026-02-21T11:16:17.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:17.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509449 2026-02-21T11:16:17.783 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 64424509449 2026-02-21T11:16:17.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-21T11:16:17.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:16:18.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509447 -lt 64424509449 2026-02-21T11:16:18.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:16:19.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:16:19.004 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:16:19.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-21T11:16:19.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:19.215 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-85899345928 2026-02-21T11:16:19.215 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:19.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:16:19.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-85899345928 2026-02-21T11:16:19.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:19.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=85899345928 2026-02-21T11:16:19.219 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 85899345928 2026-02-21T11:16:19.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 85899345928' 2026-02-21T11:16:19.219 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:16:19.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 85899345929 -lt 85899345928 2026-02-21T11:16:19.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:19.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182408 2026-02-21T11:16:19.432 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:19.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:16:19.433 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182408 2026-02-21T11:16:19.433 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:19.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182408 2026-02-21T11:16:19.435 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 107374182408 2026-02-21T11:16:19.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182408' 2026-02-21T11:16:19.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:16:19.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182408 -lt 107374182408 2026-02-21T11:16:19.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:19.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-124554051591 2026-02-21T11:16:19.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:19.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:16:19.646 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-124554051591 2026-02-21T11:16:19.646 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:19.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=124554051591 2026-02-21T11:16:19.647 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 124554051591 2026-02-21T11:16:19.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 124554051591' 2026-02-21T11:16:19.648 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:16:19.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 124554051591 -lt 124554051591 2026-02-21T11:16:19.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:19.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920774 2026-02-21T11:16:19.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:19.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:16:19.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920774 2026-02-21T11:16:19.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:19.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920774 2026-02-21T11:16:19.869 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 141733920774 2026-02-21T11:16:19.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920774' 2026-02-21T11:16:19.869 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:16:20.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-21T11:16:20.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:20.095 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789958 2026-02-21T11:16:20.095 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:20.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:16:20.097 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789958 2026-02-21T11:16:20.097 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:20.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789958 2026-02-21T11:16:20.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789958' 2026-02-21T11:16:20.099 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.7 seq 158913789958 2026-02-21T11:16:20.099 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:16:20.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789959 -lt 158913789958 2026-02-21T11:16:20.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:20.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626437 2026-02-21T11:16:20.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:20.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:16:20.313 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626437 2026-02-21T11:16:20.313 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:20.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626437 2026-02-21T11:16:20.314 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 180388626437 2026-02-21T11:16:20.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626437' 2026-02-21T11:16:20.315 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:16:20.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626437 -lt 180388626437 2026-02-21T11:16:20.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:20.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-201863462917 2026-02-21T11:16:20.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:20.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:16:20.530 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-201863462917 2026-02-21T11:16:20.530 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:20.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=201863462917 2026-02-21T11:16:20.531 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.9 seq 201863462917 2026-02-21T11:16:20.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 201863462917' 2026-02-21T11:16:20.532 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:16:20.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 201863462917 -lt 201863462917 2026-02-21T11:16:20.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:20.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332099 2026-02-21T11:16:20.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:20.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:16:20.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332099 2026-02-21T11:16:20.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:20.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332099 2026-02-21T11:16:20.754 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.10 seq 219043332099 2026-02-21T11:16:20.755 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332099' 2026-02-21T11:16:20.755 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:16:20.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332100 -lt 219043332099 2026-02-21T11:16:20.973 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:16:20.973 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:20.973 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:21.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-21T11:16:21.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:16:21.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:16:21.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:16:21.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:16:21.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:16:21.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:16:21.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:16:21.469 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-21T11:16:21.470 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:16:21.470 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:21.470 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:21.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-21T11:16:21.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:16:21.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:16:21.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-21T11:16:21.754 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:16:21.754 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:16:21.755 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:16:21.755 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:21.755 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:21.755 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46206/ceph-osd.0.asok 2026-02-21T11:16:21.755 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-21T11:16:21.755 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.46206/ceph-osd.0.asok log flush 2026-02-21T11:16:21.811 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:16:21.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-21T11:16:21.818 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:15:34.013+0000 7f59ab310a40 0 load: jerasure load: lrc 2026-02-21T11:16:21.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-21T11:16:21.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-21T11:16:21.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-21T11:16:22.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-21T11:16:22.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-21T11:16:22.473 INFO:tasks.workunit.client.0.trial166.stderr:pool 'ecpool' already exists 2026-02-21T11:16:22.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:16:23.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:16:23.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:16:23.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:16:23.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:16:23.485 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:16:23.485 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:16:23.486 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:16:23.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:16:23.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:16:23.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:16:23.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:16:23.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:16:23.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:16:23.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:16:23.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:16:23.588 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:16:23.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:23.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:16:23.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836494 2026-02-21T11:16:23.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836494 2026-02-21T11:16:23.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494' 2026-02-21T11:16:23.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:23.875 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:16:23.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672973 2026-02-21T11:16:23.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672973 2026-02-21T11:16:23.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973' 2026-02-21T11:16:23.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:23.950 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:16:24.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509453 2026-02-21T11:16:24.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509453 2026-02-21T11:16:24.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453' 2026-02-21T11:16:24.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:16:24.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=85899345932 2026-02-21T11:16:24.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 85899345932 2026-02-21T11:16:24.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932' 2026-02-21T11:16:24.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:16:24.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182411 2026-02-21T11:16:24.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182411 2026-02-21T11:16:24.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411' 2026-02-21T11:16:24.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.189 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:16:24.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=124554051594 2026-02-21T11:16:24.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 124554051594 2026-02-21T11:16:24.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411 5-124554051594' 2026-02-21T11:16:24.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.267 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:16:24.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920777 2026-02-21T11:16:24.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920777 2026-02-21T11:16:24.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411 5-124554051594 6-141733920777' 2026-02-21T11:16:24.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.348 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:16:24.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789962 2026-02-21T11:16:24.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789962 2026-02-21T11:16:24.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411 5-124554051594 6-141733920777 7-158913789962' 2026-02-21T11:16:24.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.428 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:16:24.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626440 2026-02-21T11:16:24.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626440 2026-02-21T11:16:24.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411 5-124554051594 6-141733920777 7-158913789962 8-180388626440' 2026-02-21T11:16:24.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:16:24.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=201863462920 2026-02-21T11:16:24.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 201863462920 2026-02-21T11:16:24.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411 5-124554051594 6-141733920777 7-158913789962 8-180388626440 9-201863462920' 2026-02-21T11:16:24.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:24.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:16:24.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332103 2026-02-21T11:16:24.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332103 2026-02-21T11:16:24.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-42949672973 2-64424509453 3-85899345932 4-107374182411 5-124554051594 6-141733920777 7-158913789962 8-180388626440 9-201863462920 10-219043332103' 2026-02-21T11:16:24.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:24.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836494 2026-02-21T11:16:24.665 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:24.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:16:24.666 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836494 2026-02-21T11:16:24.666 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:24.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836494 2026-02-21T11:16:24.667 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836494 2026-02-21T11:16:24.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-21T11:16:24.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:24.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-21T11:16:24.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:24.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672973 2026-02-21T11:16:24.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:24.877 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:16:24.879 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672973 2026-02-21T11:16:24.879 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:24.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672973 2026-02-21T11:16:24.880 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 42949672973 2026-02-21T11:16:24.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-21T11:16:24.881 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:16:25.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672974 -lt 42949672973 2026-02-21T11:16:25.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:25.103 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509453 2026-02-21T11:16:25.103 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:25.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:16:25.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509453 2026-02-21T11:16:25.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:25.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509453 2026-02-21T11:16:25.106 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 64424509453 2026-02-21T11:16:25.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509453' 2026-02-21T11:16:25.106 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:16:25.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509453 -lt 64424509453 2026-02-21T11:16:25.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:25.329 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-85899345932 2026-02-21T11:16:25.329 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:25.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:16:25.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-85899345932 2026-02-21T11:16:25.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:25.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=85899345932 2026-02-21T11:16:25.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 85899345932' 2026-02-21T11:16:25.333 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 85899345932 2026-02-21T11:16:25.333 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:16:25.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 85899345932 -lt 85899345932 2026-02-21T11:16:25.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:25.561 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182411 2026-02-21T11:16:25.561 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:25.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:16:25.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182411 2026-02-21T11:16:25.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:25.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182411 2026-02-21T11:16:25.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182411' 2026-02-21T11:16:25.565 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 107374182411 2026-02-21T11:16:25.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:16:25.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182411 -lt 107374182411 2026-02-21T11:16:25.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:25.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-124554051594 2026-02-21T11:16:25.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:25.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:16:25.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-124554051594 2026-02-21T11:16:25.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:25.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=124554051594 2026-02-21T11:16:25.796 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 124554051594 2026-02-21T11:16:25.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 124554051594' 2026-02-21T11:16:25.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:16:26.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 124554051594 -lt 124554051594 2026-02-21T11:16:26.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:26.015 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920777 2026-02-21T11:16:26.015 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:26.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:16:26.017 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920777 2026-02-21T11:16:26.017 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:26.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920777 2026-02-21T11:16:26.018 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 141733920777 2026-02-21T11:16:26.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920777' 2026-02-21T11:16:26.019 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:16:26.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920778 -lt 141733920777 2026-02-21T11:16:26.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:26.237 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789962 2026-02-21T11:16:26.237 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:26.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:16:26.239 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789962 2026-02-21T11:16:26.239 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:26.240 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789962 2026-02-21T11:16:26.240 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789962' 2026-02-21T11:16:26.240 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.7 seq 158913789962 2026-02-21T11:16:26.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:16:26.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789962 -lt 158913789962 2026-02-21T11:16:26.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:26.469 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626440 2026-02-21T11:16:26.469 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:26.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:16:26.470 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626440 2026-02-21T11:16:26.471 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:26.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626440 2026-02-21T11:16:26.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626440' 2026-02-21T11:16:26.472 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 180388626440 2026-02-21T11:16:26.473 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:16:26.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626440 -lt 180388626440 2026-02-21T11:16:26.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:26.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-201863462920 2026-02-21T11:16:26.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:16:26.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-201863462920 2026-02-21T11:16:26.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:26.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=201863462920 2026-02-21T11:16:26.684 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.9 seq 201863462920 2026-02-21T11:16:26.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 201863462920' 2026-02-21T11:16:26.684 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:16:26.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 201863462921 -lt 201863462920 2026-02-21T11:16:26.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:26.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332103 2026-02-21T11:16:26.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:26.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:16:26.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332103 2026-02-21T11:16:26.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:26.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332103 2026-02-21T11:16:26.902 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.10 seq 219043332103 2026-02-21T11:16:26.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332103' 2026-02-21T11:16:26.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:16:27.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332103 -lt 219043332103 2026-02-21T11:16:27.121 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:16:27.121 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:27.121 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:27.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T11:16:27.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:16:27.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:16:27.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:16:27.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:16:27.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:16:27.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:16:27.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:16:27.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T11:16:27.622 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:16:27.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:27.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:27.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T11:16:27.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:16:27.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:16:27.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-21T11:16:27.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T11:16:27.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:TEST_chunk_mapping 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_put_get_isa 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_put_get_jerasure 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_put_get_lrc_advanced 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_put_get_lrc_kml 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_put_get_shec' 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: local payload=ABC 2026-02-21T11:16:27.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:267: TEST_alignment_constraints: echo ABC 2026-02-21T11:16:27.915 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: chunk_size 2026-02-21T11:16:27.915 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-21T11:16:27.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-21T11:16:27.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-21T11:16:27.927 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-21T11:16:27.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-21T11:16:27.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-21T11:16:27.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-21T11:16:27.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-21T11:16:27.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: local stripe_unit=4096 2026-02-21T11:16:27.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-21T11:16:27.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: grep k= 2026-02-21T11:16:28.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: eval local k=2 2026-02-21T11:16:28.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: local k=2 2026-02-21T11:16:28.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:275: TEST_alignment_constraints: local block_size=8191 2026-02-21T11:16:28.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:276: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-21T11:16:28.158 INFO:tasks.workunit.client.0.trial166.stderr:2+0 records in 2026-02-21T11:16:28.158 INFO:tasks.workunit.client.0.trial166.stderr:2+0 records out 2026-02-21T11:16:28.158 INFO:tasks.workunit.client.0.trial166.stderr:16382 bytes (16 kB, 16 KiB) copied, 0.000124349 s, 132 MB/s 2026-02-21T11:16:28.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:277: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-21T11:16:28.180 INFO:tasks.workunit.client.0.trial166.stderr:INFO: op_size has been rounded to 8192 2026-02-21T11:16:28.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:279: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:16:28.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:16:28.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:320: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=ecpool 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=0 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=1 2026-02-21T11:16:28.187 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-21T11:16:28.188 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-21T11:16:28.196 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-21T11:16:28.197 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-21T11:16:28.197 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-21T11:16:28.213 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-21T11:16:28.213 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-21T11:16:28.213 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-21T11:16:28.213 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-21T11:16:28.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-21T11:16:28.213 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-21T11:16:28.214 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-21T11:16:28.222 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-21T11:16:28.222 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-21T11:16:28.222 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-21T11:16:28.233 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-21T11:16:28.233 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-21T11:16:28.233 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-21T11:16:28.233 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-21T11:16:28.233 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-21T11:16:28.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-21T11:16:28.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-21T11:16:28.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-21T11:16:28.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-21T11:16:28.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-21T11:16:28.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-21T11:16:28.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-21T11:16:28.286 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-21T11:16:28.287 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:16:28.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:0' 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-21T11:16:28.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-21T11:16:28.559 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:28.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:28.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-21T11:16:28.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-21T11:16:28.624 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:28.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:28.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-21T11:16:28.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-21T11:16:28.698 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:28.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:28.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-21T11:16:28.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-21T11:16:28.772 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:28.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:28.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-21T11:16:28.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:16:28.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-21T11:16:28.777 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-21T11:16:28.777 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-21T11:16:28.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-21T11:16:28.778 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-21T11:16:28.778 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:16:28.997 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:0' 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:16:28.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:16:28.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:16:29.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-21T11:16:29.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:29.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:29.490 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:16:29.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:16:29.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-21T11:16:29.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T11:16:29.492 INFO:tasks.workunit.client.0.trial166.stderr:start osd.3 2026-02-21T11:16:29.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:29.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-21T11:16:29.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T11:16:29.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:16:29.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:16:29.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:16:29.513 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:29.508+0000 7f83b9b68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:29.514 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:29.512+0000 7f83b9b68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:29.515 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:29.512+0000 7f83b9b68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:29.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T11:16:29.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:29.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:29.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:16:29.839 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:29.835+0000 7f83b9b68a40 -1 Falling back to public interface 2026-02-21T11:16:29.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:30.541 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:30.535+0000 7f83b9b68a40 -1 osd.3 65 log_to_monitors true 2026-02-21T11:16:30.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:30.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:30.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:30.909 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:30.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:30.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 up in weight 1 up_from 69 up_thru 69 down_at 66 last_clean_interval [20,65) [v2:127.0.0.1:6826/3007350898,v1:127.0.0.1:6827/3007350898] [v2:127.0.0.1:6828/3007350898,v1:127.0.0.1:6829/3007350898] exists,up 1b5515ea-f993-49ac-a875-4b34cfde9205 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:16:31.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:16:31.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:16:31.123 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:16:31.123 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:16:31.124 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:16:31.124 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:16:31.124 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:16:31.124 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:16:31.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:16:31.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:16:31.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:16:31.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:16:31.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:16:31.219 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:16:31.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:16:31.431 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:31.431 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.432 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:16:31.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836498 2026-02-21T11:16:31.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836498 2026-02-21T11:16:31.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498' 2026-02-21T11:16:31.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.512 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:16:31.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672977 2026-02-21T11:16:31.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672977 2026-02-21T11:16:31.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977' 2026-02-21T11:16:31.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:16:31.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509456 2026-02-21T11:16:31.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509456 2026-02-21T11:16:31.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456' 2026-02-21T11:16:31.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.674 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:16:31.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743427 2026-02-21T11:16:31.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743427 2026-02-21T11:16:31.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427' 2026-02-21T11:16:31.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:16:31.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182415 2026-02-21T11:16:31.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182415 2026-02-21T11:16:31.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415' 2026-02-21T11:16:31.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:16:31.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=124554051598 2026-02-21T11:16:31.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 124554051598 2026-02-21T11:16:31.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415 5-124554051598' 2026-02-21T11:16:31.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.906 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:16:31.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920781 2026-02-21T11:16:31.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920781 2026-02-21T11:16:31.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415 5-124554051598 6-141733920781' 2026-02-21T11:16:31.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:31.984 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:16:32.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789965 2026-02-21T11:16:32.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789965 2026-02-21T11:16:32.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415 5-124554051598 6-141733920781 7-158913789965' 2026-02-21T11:16:32.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:32.064 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:16:32.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626444 2026-02-21T11:16:32.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626444 2026-02-21T11:16:32.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415 5-124554051598 6-141733920781 7-158913789965 8-180388626444' 2026-02-21T11:16:32.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:32.145 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:16:32.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=201863462924 2026-02-21T11:16:32.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 201863462924 2026-02-21T11:16:32.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415 5-124554051598 6-141733920781 7-158913789965 8-180388626444 9-201863462924' 2026-02-21T11:16:32.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:32.226 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:16:32.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332106 2026-02-21T11:16:32.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332106 2026-02-21T11:16:32.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-42949672977 2-64424509456 3-296352743427 4-107374182415 5-124554051598 6-141733920781 7-158913789965 8-180388626444 9-201863462924 10-219043332106' 2026-02-21T11:16:32.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:32.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836498 2026-02-21T11:16:32.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:32.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:16:32.305 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836498 2026-02-21T11:16:32.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:32.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836498 2026-02-21T11:16:32.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-21T11:16:32.307 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 21474836498 2026-02-21T11:16:32.307 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836495 -lt 21474836498 2026-02-21T11:16:32.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:16:33.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:16:33.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:33.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-21T11:16:33.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:33.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672977 2026-02-21T11:16:33.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:33.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:16:33.734 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672977 2026-02-21T11:16:33.734 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:33.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672977 2026-02-21T11:16:33.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672977' 2026-02-21T11:16:33.736 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 42949672977 2026-02-21T11:16:33.736 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:16:33.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672977 -lt 42949672977 2026-02-21T11:16:33.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:33.957 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509456 2026-02-21T11:16:33.957 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:33.958 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:16:33.959 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509456 2026-02-21T11:16:33.959 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:33.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509456 2026-02-21T11:16:33.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509456' 2026-02-21T11:16:33.960 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 64424509456 2026-02-21T11:16:33.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:16:34.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509456 -lt 64424509456 2026-02-21T11:16:34.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:34.181 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743427 2026-02-21T11:16:34.181 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:34.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:16:34.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743427 2026-02-21T11:16:34.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:34.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743427 2026-02-21T11:16:34.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743427' 2026-02-21T11:16:34.184 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 296352743427 2026-02-21T11:16:34.184 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:16:34.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743427 -lt 296352743427 2026-02-21T11:16:34.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:34.403 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182415 2026-02-21T11:16:34.403 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:34.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:16:34.404 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182415 2026-02-21T11:16:34.404 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:34.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182415 2026-02-21T11:16:34.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182415' 2026-02-21T11:16:34.406 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.4 seq 107374182415 2026-02-21T11:16:34.406 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:16:34.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182415 -lt 107374182415 2026-02-21T11:16:34.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:34.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-124554051598 2026-02-21T11:16:34.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:34.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:16:34.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-124554051598 2026-02-21T11:16:34.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:34.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=124554051598 2026-02-21T11:16:34.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 124554051598' 2026-02-21T11:16:34.637 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.5 seq 124554051598 2026-02-21T11:16:34.637 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:16:34.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 124554051598 -lt 124554051598 2026-02-21T11:16:34.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:34.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920781 2026-02-21T11:16:34.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:34.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:16:34.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920781 2026-02-21T11:16:34.861 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:34.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920781 2026-02-21T11:16:34.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920781' 2026-02-21T11:16:34.862 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.6 seq 141733920781 2026-02-21T11:16:34.863 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:16:35.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920782 -lt 141733920781 2026-02-21T11:16:35.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:35.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789965 2026-02-21T11:16:35.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:35.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:16:35.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789965 2026-02-21T11:16:35.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:35.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789965 2026-02-21T11:16:35.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789965' 2026-02-21T11:16:35.084 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.7 seq 158913789965 2026-02-21T11:16:35.084 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:16:35.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789966 -lt 158913789965 2026-02-21T11:16:35.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:35.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626444 2026-02-21T11:16:35.307 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:35.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:16:35.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626444 2026-02-21T11:16:35.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:35.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626444 2026-02-21T11:16:35.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626444' 2026-02-21T11:16:35.311 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.8 seq 180388626444 2026-02-21T11:16:35.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:16:35.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626444 -lt 180388626444 2026-02-21T11:16:35.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:35.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-201863462924 2026-02-21T11:16:35.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:35.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:16:35.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-201863462924 2026-02-21T11:16:35.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:35.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=201863462924 2026-02-21T11:16:35.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 201863462924' 2026-02-21T11:16:35.522 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.9 seq 201863462924 2026-02-21T11:16:35.523 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:16:35.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 201863462924 -lt 201863462924 2026-02-21T11:16:35.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:35.734 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332106 2026-02-21T11:16:35.735 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:35.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:16:35.736 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332106 2026-02-21T11:16:35.736 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:35.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332106 2026-02-21T11:16:35.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332106' 2026-02-21T11:16:35.738 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.10 seq 219043332106 2026-02-21T11:16:35.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:16:35.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332107 -lt 219043332106 2026-02-21T11:16:35.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:16:35.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:35.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:36.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T11:16:36.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:16:36.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:16:36.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:16:36.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:16:36.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:16:36.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:16:36.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:16:36.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T11:16:36.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:16:36.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:36.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:36.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T11:16:36.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:16:36.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:16:36.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:16:36.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:16:37.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:16:37.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-21T11:16:37.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-21T11:16:37.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:16:37.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-21T11:16:37.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:16:37.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-21T11:16:37.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-21T11:16:37.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-21T11:16:37.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-21T11:16:37.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:37.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:16:37.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-21T11:16:37.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-21T11:16:37.197 INFO:tasks.workunit.client.0.trial166.stderr:start osd.5 2026-02-21T11:16:37.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:37.197 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-21T11:16:37.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-21T11:16:37.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:16:37.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:16:37.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:16:37.213 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:37.207+0000 7f5705530a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:37.214 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:37.211+0000 7f5705530a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:37.215 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:37.211+0000 7f5705530a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:37.402 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:16:37.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:37.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:16:37.592 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:37.587+0000 7f5705530a40 -1 Falling back to public interface 2026-02-21T11:16:37.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:38.258 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:38.255+0000 7f5705530a40 -1 osd.5 71 log_to_monitors true 2026-02-21T11:16:38.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:38.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:38.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:38.620 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:38.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:38.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:16:38.836 INFO:tasks.workunit.client.0.trial166.stderr:osd.5 up in weight 1 up_from 75 up_thru 75 down_at 72 last_clean_interval [29,71) [v2:127.0.0.1:6842/2610630517,v1:127.0.0.1:6843/2610630517] [v2:127.0.0.1:6844/2610630517,v1:127.0.0.1:6845/2610630517] exists,up cc03670a-b315-45a4-bd4f-afa125bcdeb6 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:16:38.837 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:16:38.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:16:38.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:16:38.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:16:38.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:16:38.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:16:38.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:16:38.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:16:38.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:16:38.935 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:16:39.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:16:39.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:16:39.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:16:39.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836501 2026-02-21T11:16:39.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836501 2026-02-21T11:16:39.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501' 2026-02-21T11:16:39.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:16:39.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672980 2026-02-21T11:16:39.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672980 2026-02-21T11:16:39.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980' 2026-02-21T11:16:39.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:16:39.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509460 2026-02-21T11:16:39.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509460 2026-02-21T11:16:39.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460' 2026-02-21T11:16:39.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.394 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:16:39.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743430 2026-02-21T11:16:39.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743430 2026-02-21T11:16:39.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430' 2026-02-21T11:16:39.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:16:39.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182418 2026-02-21T11:16:39.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182418 2026-02-21T11:16:39.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418' 2026-02-21T11:16:39.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.552 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:16:39.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=322122547203 2026-02-21T11:16:39.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 322122547203 2026-02-21T11:16:39.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418 5-322122547203' 2026-02-21T11:16:39.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:16:39.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920785 2026-02-21T11:16:39.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920785 2026-02-21T11:16:39.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418 5-322122547203 6-141733920785' 2026-02-21T11:16:39.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:16:39.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789969 2026-02-21T11:16:39.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789969 2026-02-21T11:16:39.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418 5-322122547203 6-141733920785 7-158913789969' 2026-02-21T11:16:39.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:16:39.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626447 2026-02-21T11:16:39.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626447 2026-02-21T11:16:39.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418 5-322122547203 6-141733920785 7-158913789969 8-180388626447' 2026-02-21T11:16:39.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:16:39.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=201863462927 2026-02-21T11:16:39.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 201863462927 2026-02-21T11:16:39.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418 5-322122547203 6-141733920785 7-158913789969 8-180388626447 9-201863462927' 2026-02-21T11:16:39.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:39.954 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:16:40.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332110 2026-02-21T11:16:40.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332110 2026-02-21T11:16:40.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-42949672980 2-64424509460 3-296352743430 4-107374182418 5-322122547203 6-141733920785 7-158913789969 8-180388626447 9-201863462927 10-219043332110' 2026-02-21T11:16:40.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:40.031 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836501 2026-02-21T11:16:40.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:40.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:16:40.033 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836501 2026-02-21T11:16:40.033 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:40.035 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836501 2026-02-21T11:16:40.035 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-21T11:16:40.035 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 21474836501 2026-02-21T11:16:40.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:40.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836499 -lt 21474836501 2026-02-21T11:16:40.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:16:41.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:16:41.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:41.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-21T11:16:41.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:41.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672980 2026-02-21T11:16:41.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:41.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:16:41.469 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672980 2026-02-21T11:16:41.469 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:41.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672980 2026-02-21T11:16:41.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672980' 2026-02-21T11:16:41.471 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 42949672980 2026-02-21T11:16:41.471 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:16:41.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672981 -lt 42949672980 2026-02-21T11:16:41.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:41.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509460 2026-02-21T11:16:41.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:41.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:16:41.691 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509460 2026-02-21T11:16:41.691 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:41.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509460 2026-02-21T11:16:41.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509460' 2026-02-21T11:16:41.693 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 64424509460 2026-02-21T11:16:41.693 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:16:41.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509460 -lt 64424509460 2026-02-21T11:16:41.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:41.918 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743430 2026-02-21T11:16:41.918 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:41.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:16:41.919 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743430 2026-02-21T11:16:41.919 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:41.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743430 2026-02-21T11:16:41.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743430' 2026-02-21T11:16:41.921 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 296352743430 2026-02-21T11:16:41.921 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:16:42.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743431 -lt 296352743430 2026-02-21T11:16:42.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:42.142 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182418 2026-02-21T11:16:42.142 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:42.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:16:42.144 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182418 2026-02-21T11:16:42.144 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:42.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182418 2026-02-21T11:16:42.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182418' 2026-02-21T11:16:42.146 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.4 seq 107374182418 2026-02-21T11:16:42.146 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:16:42.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182418 -lt 107374182418 2026-02-21T11:16:42.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:42.370 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-322122547203 2026-02-21T11:16:42.370 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:42.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:16:42.372 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-322122547203 2026-02-21T11:16:42.372 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:42.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=322122547203 2026-02-21T11:16:42.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 322122547203' 2026-02-21T11:16:42.375 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.5 seq 322122547203 2026-02-21T11:16:42.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:16:42.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 322122547203 -lt 322122547203 2026-02-21T11:16:42.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:42.604 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920785 2026-02-21T11:16:42.604 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:42.606 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:16:42.606 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920785 2026-02-21T11:16:42.606 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:42.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920785 2026-02-21T11:16:42.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920785' 2026-02-21T11:16:42.607 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.6 seq 141733920785 2026-02-21T11:16:42.607 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:16:42.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920785 -lt 141733920785 2026-02-21T11:16:42.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:42.814 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789969 2026-02-21T11:16:42.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:42.815 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:16:42.816 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789969 2026-02-21T11:16:42.816 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:42.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789969 2026-02-21T11:16:42.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789969' 2026-02-21T11:16:42.818 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.7 seq 158913789969 2026-02-21T11:16:42.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:16:43.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789970 -lt 158913789969 2026-02-21T11:16:43.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:43.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626447 2026-02-21T11:16:43.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:43.035 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:16:43.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626447 2026-02-21T11:16:43.035 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:43.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626447 2026-02-21T11:16:43.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626447' 2026-02-21T11:16:43.037 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.8 seq 180388626447 2026-02-21T11:16:43.037 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:16:43.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626448 -lt 180388626447 2026-02-21T11:16:43.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:43.256 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-201863462927 2026-02-21T11:16:43.256 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:43.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:16:43.258 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-201863462927 2026-02-21T11:16:43.258 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:43.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=201863462927 2026-02-21T11:16:43.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 201863462927' 2026-02-21T11:16:43.259 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.9 seq 201863462927 2026-02-21T11:16:43.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:16:43.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 201863462928 -lt 201863462927 2026-02-21T11:16:43.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:43.480 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332110 2026-02-21T11:16:43.480 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:43.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:16:43.482 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332110 2026-02-21T11:16:43.482 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:43.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332110 2026-02-21T11:16:43.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332110' 2026-02-21T11:16:43.483 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.10 seq 219043332110 2026-02-21T11:16:43.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:16:43.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332110 -lt 219043332110 2026-02-21T11:16:43.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:16:43.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:43.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:44.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:16:44.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:16:44.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T11:16:44.212 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:16:44.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:44.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:44.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T11:16:44.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:16:44.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:16:44.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:322: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-21T11:16:44.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-21T11:16:44.990 INFO:tasks.workunit.client.0.trial166.stdout:crush-device-class= 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:crush-failure-domain=host 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:crush-num-failure-domains=0 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:crush-osds-per-failure-domain=0 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:crush-root=default 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:k=-1 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:l=-1 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:layers=[ [ "cDD", "" ] ] 2026-02-21T11:16:44.991 INFO:tasks.workunit.client.0.trial166.stdout:m=-1 2026-02-21T11:16:44.992 INFO:tasks.workunit.client.0.trial166.stdout:mapping=_DD 2026-02-21T11:16:44.992 INFO:tasks.workunit.client.0.trial166.stdout:plugin=lrc 2026-02-21T11:16:45.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:328: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-21T11:16:45.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-21T11:16:45.342 INFO:tasks.workunit.client.0.trial166.stderr:pool 'remap-pool' already exists 2026-02-21T11:16:45.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:16:46.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:336: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-21T11:16:46.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-21T11:16:46.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=remap-pool 2026-02-21T11:16:46.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=1 2026-02-21T11:16:46.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=2 2026-02-21T11:16:46.356 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-21T11:16:46.356 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-21T11:16:46.367 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-21T11:16:46.367 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-21T11:16:46.368 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-21T11:16:46.383 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-21T11:16:46.383 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-21T11:16:46.383 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-21T11:16:46.383 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-21T11:16:46.383 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-21T11:16:46.384 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-21T11:16:46.384 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-21T11:16:46.393 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-21T11:16:46.393 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-21T11:16:46.393 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-21T11:16:46.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-21T11:16:46.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-21T11:16:46.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-21T11:16:46.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-21T11:16:46.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-21T11:16:46.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-21T11:16:46.457 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-21T11:16:46.457 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:16:46.671 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:9' 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-21T11:16:46.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-21T11:16:46.738 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:46.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:46.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-21T11:16:46.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-21T11:16:46.815 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:46.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:46.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-21T11:16:46.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-21T11:16:46.888 INFO:tasks.workunit.client.0.trial166.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-21T11:16:46.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-21T11:16:46.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-21T11:16:46.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:16:46.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-21T11:16:46.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-21T11:16:46.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-21T11:16:46.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-21T11:16:46.893 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-21T11:16:46.893 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr:9' 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-21T11:16:47.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:16:47.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-21T11:16:47.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-21T11:16:47.114 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:16:47.114 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:16:47.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:16:47.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:16:47.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:16:47.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:16:47.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:47.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-21T11:16:47.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:16:47.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-21T11:16:47.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:16:47.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-21T11:16:47.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:47.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:47.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:16:47.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:47.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-21T11:16:47.625 INFO:tasks.workunit.client.0.trial166.stderr:start osd.6 2026-02-21T11:16:47.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:47.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-21T11:16:47.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-21T11:16:47.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:16:47.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:16:47.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:16:47.636 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:47.631+0000 7f094764ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:47.638 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:47.635+0000 7f094764ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:47.639 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:47.635+0000 7f094764ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:47.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-21T11:16:47.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:47.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:16:48.012 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:48.007+0000 7f094764ba40 -1 Falling back to public interface 2026-02-21T11:16:48.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:48.656 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:48.651+0000 7f094764ba40 -1 osd.6 86 log_to_monitors true 2026-02-21T11:16:49.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:49.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:49.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:49.057 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:49.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:49.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:16:49.268 INFO:tasks.workunit.client.0.trial166.stderr:osd.6 up in weight 1 up_from 90 up_thru 90 down_at 87 last_clean_interval [33,86) [v2:127.0.0.1:6850/2470357247,v1:127.0.0.1:6851/2470357247] [v2:127.0.0.1:6852/2470357247,v1:127.0.0.1:6853/2470357247] exists,up 1810de0f-541f-419e-9ac7-adfe593930ea 2026-02-21T11:16:49.268 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:16:49.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:16:49.270 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:16:49.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:16:49.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:16:49.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:16:49.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:16:49.373 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:16:49.373 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:49.584 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:16:49.585 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:16:49.585 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:49.585 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:16:49.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:16:49.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:49.585 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:16:49.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836505 2026-02-21T11:16:49.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836505 2026-02-21T11:16:49.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505' 2026-02-21T11:16:49.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:49.665 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:16:49.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672985 2026-02-21T11:16:49.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672985 2026-02-21T11:16:49.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985' 2026-02-21T11:16:49.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:49.744 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:16:49.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509464 2026-02-21T11:16:49.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509464 2026-02-21T11:16:49.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464' 2026-02-21T11:16:49.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:49.824 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:16:49.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743434 2026-02-21T11:16:49.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743434 2026-02-21T11:16:49.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434' 2026-02-21T11:16:49.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:49.905 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:16:49.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182422 2026-02-21T11:16:49.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182422 2026-02-21T11:16:49.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422' 2026-02-21T11:16:49.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:49.986 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:16:50.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=322122547207 2026-02-21T11:16:50.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 322122547207 2026-02-21T11:16:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422 5-322122547207' 2026-02-21T11:16:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:50.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:16:50.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=386547056643 2026-02-21T11:16:50.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 386547056643 2026-02-21T11:16:50.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422 5-322122547207 6-386547056643' 2026-02-21T11:16:50.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:50.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:16:50.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789973 2026-02-21T11:16:50.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789973 2026-02-21T11:16:50.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422 5-322122547207 6-386547056643 7-158913789973' 2026-02-21T11:16:50.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:50.223 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:16:50.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626451 2026-02-21T11:16:50.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626451 2026-02-21T11:16:50.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422 5-322122547207 6-386547056643 7-158913789973 8-180388626451' 2026-02-21T11:16:50.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:50.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:16:50.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=201863462931 2026-02-21T11:16:50.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 201863462931 2026-02-21T11:16:50.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422 5-322122547207 6-386547056643 7-158913789973 8-180388626451 9-201863462931' 2026-02-21T11:16:50.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:50.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:16:50.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332114 2026-02-21T11:16:50.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332114 2026-02-21T11:16:50.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-42949672985 2-64424509464 3-296352743434 4-107374182422 5-322122547207 6-386547056643 7-158913789973 8-180388626451 9-201863462931 10-219043332114' 2026-02-21T11:16:50.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:50.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836505 2026-02-21T11:16:50.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:50.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:16:50.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836505 2026-02-21T11:16:50.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:50.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836505 2026-02-21T11:16:50.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-21T11:16:50.467 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 21474836505 2026-02-21T11:16:50.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:50.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836503 -lt 21474836505 2026-02-21T11:16:50.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:16:51.673 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:16:51.673 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:51.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836505 -lt 21474836505 2026-02-21T11:16:51.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:51.883 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672985 2026-02-21T11:16:51.883 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:51.884 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:16:51.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672985 2026-02-21T11:16:51.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:51.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672985 2026-02-21T11:16:51.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672985' 2026-02-21T11:16:51.886 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 42949672985 2026-02-21T11:16:51.887 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:16:52.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672985 -lt 42949672985 2026-02-21T11:16:52.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:52.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509464 2026-02-21T11:16:52.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:52.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:16:52.107 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509464 2026-02-21T11:16:52.107 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:52.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509464 2026-02-21T11:16:52.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509464' 2026-02-21T11:16:52.108 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 64424509464 2026-02-21T11:16:52.109 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:16:52.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509464 -lt 64424509464 2026-02-21T11:16:52.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:52.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743434 2026-02-21T11:16:52.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:52.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:16:52.332 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743434 2026-02-21T11:16:52.333 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:52.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743434 2026-02-21T11:16:52.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743434' 2026-02-21T11:16:52.334 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 296352743434 2026-02-21T11:16:52.334 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:16:52.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743435 -lt 296352743434 2026-02-21T11:16:52.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:52.554 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182422 2026-02-21T11:16:52.554 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:52.555 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:16:52.556 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182422 2026-02-21T11:16:52.556 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:52.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182422 2026-02-21T11:16:52.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182422' 2026-02-21T11:16:52.557 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.4 seq 107374182422 2026-02-21T11:16:52.558 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:16:52.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182423 -lt 107374182422 2026-02-21T11:16:52.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:52.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-322122547207 2026-02-21T11:16:52.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:52.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:16:52.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-322122547207 2026-02-21T11:16:52.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:52.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=322122547207 2026-02-21T11:16:52.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 322122547207' 2026-02-21T11:16:52.782 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.5 seq 322122547207 2026-02-21T11:16:52.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:16:53.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 322122547207 -lt 322122547207 2026-02-21T11:16:53.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:53.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-386547056643 2026-02-21T11:16:53.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:53.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:16:53.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-386547056643 2026-02-21T11:16:53.008 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:53.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=386547056643 2026-02-21T11:16:53.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 386547056643' 2026-02-21T11:16:53.009 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.6 seq 386547056643 2026-02-21T11:16:53.009 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:16:53.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-21T11:16:53.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:53.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789973 2026-02-21T11:16:53.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:53.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:16:53.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789973 2026-02-21T11:16:53.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:53.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789973 2026-02-21T11:16:53.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789973' 2026-02-21T11:16:53.236 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.7 seq 158913789973 2026-02-21T11:16:53.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:16:53.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789974 -lt 158913789973 2026-02-21T11:16:53.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:53.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626451 2026-02-21T11:16:53.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:53.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:16:53.457 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626451 2026-02-21T11:16:53.457 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626451 2026-02-21T11:16:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626451' 2026-02-21T11:16:53.459 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.8 seq 180388626451 2026-02-21T11:16:53.459 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:16:53.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626452 -lt 180388626451 2026-02-21T11:16:53.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:53.676 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-201863462931 2026-02-21T11:16:53.676 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:53.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:16:53.678 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-201863462931 2026-02-21T11:16:53.678 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:53.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=201863462931 2026-02-21T11:16:53.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 201863462931' 2026-02-21T11:16:53.679 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.9 seq 201863462931 2026-02-21T11:16:53.679 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:16:53.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 201863462932 -lt 201863462931 2026-02-21T11:16:53.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:53.891 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332114 2026-02-21T11:16:53.891 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:53.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:16:53.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332114 2026-02-21T11:16:53.893 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:53.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332114 2026-02-21T11:16:53.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332114' 2026-02-21T11:16:53.894 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.10 seq 219043332114 2026-02-21T11:16:53.894 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:16:54.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332114 -lt 219043332114 2026-02-21T11:16:54.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:16:54.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:54.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:54.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-21T11:16:54.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:16:54.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:16:54.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:16:54.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:16:54.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:16:54.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:16:54.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:16:54.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-21T11:16:54.615 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:16:54.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:16:54.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:16:54.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-21T11:16:54.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:16:54.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:16:54.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:54.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-21T11:16:54.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:16:54.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-21T11:16:54.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:16:54.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:16:55.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-21T11:16:55.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-21T11:16:55.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:16:55.358 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:16:55.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:16:55.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:16:55.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-21T11:16:55.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-21T11:16:55.360 INFO:tasks.workunit.client.0.trial166.stderr:start osd.9 2026-02-21T11:16:55.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=f6799c1e-2b7f-4579-bb59-66d22419601a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46206/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:16:55.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-21T11:16:55.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-21T11:16:55.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:16:55.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:16:55.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:16:55.375 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:55.371+0000 7fb334e39a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:55.377 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:55.371+0000 7fb334e39a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:55.378 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:55.371+0000 7fb334e39a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:16:55.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-21T11:16:55.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:16:55.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-21T11:16:55.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:16:55.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:16:55.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:55.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:16:55.575 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:16:55.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:55.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-21T11:16:55.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:16:55.796 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:55.791+0000 7fb334e39a40 -1 Falling back to public interface 2026-02-21T11:16:56.422 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:16:56.415+0000 7fb334e39a40 -1 osd.9 93 log_to_monitors true 2026-02-21T11:16:56.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:16:56.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:16:56.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:16:56.789 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:56.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:16:56.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-21T11:16:56.991 INFO:tasks.workunit.client.0.trial166.stderr:osd.9 up in weight 1 up_from 96 up_thru 96 down_at 93 last_clean_interval [47,93) [v2:127.0.0.1:6874/82889787,v1:127.0.0.1:6875/82889787] [v2:127.0.0.1:6876/82889787,v1:127.0.0.1:6877/82889787] exists,up af42a358-deac-40d5-ba47-33a4947c7d7d 2026-02-21T11:16:56.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:16:56.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:16:56.992 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:16:56.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:16:56.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:16:56.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:16:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:16:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:16:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:16:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:16:57.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:16:57.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:16:57.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:16:57.305 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:16:57.305 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:16:57.305 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:16:57.305 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:16:57.305 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:16:57.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836509 2026-02-21T11:16:57.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836509 2026-02-21T11:16:57.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509' 2026-02-21T11:16:57.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:16:57.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672988 2026-02-21T11:16:57.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672988 2026-02-21T11:16:57.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988' 2026-02-21T11:16:57.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:16:57.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509467 2026-02-21T11:16:57.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509467 2026-02-21T11:16:57.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467' 2026-02-21T11:16:57.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.549 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:16:57.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743438 2026-02-21T11:16:57.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743438 2026-02-21T11:16:57.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438' 2026-02-21T11:16:57.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:16:57.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182426 2026-02-21T11:16:57.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182426 2026-02-21T11:16:57.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426' 2026-02-21T11:16:57.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:16:57.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=322122547210 2026-02-21T11:16:57.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 322122547210 2026-02-21T11:16:57.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426 5-322122547210' 2026-02-21T11:16:57.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.786 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:16:57.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=386547056646 2026-02-21T11:16:57.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 386547056646 2026-02-21T11:16:57.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426 5-322122547210 6-386547056646' 2026-02-21T11:16:57.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:16:57.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789977 2026-02-21T11:16:57.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789977 2026-02-21T11:16:57.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426 5-322122547210 6-386547056646 7-158913789977' 2026-02-21T11:16:57.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:57.943 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:16:58.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626455 2026-02-21T11:16:58.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626455 2026-02-21T11:16:58.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426 5-322122547210 6-386547056646 7-158913789977 8-180388626455' 2026-02-21T11:16:58.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:58.021 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:16:58.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=412316860419 2026-02-21T11:16:58.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 412316860419 2026-02-21T11:16:58.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426 5-322122547210 6-386547056646 7-158913789977 8-180388626455 9-412316860419' 2026-02-21T11:16:58.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:16:58.099 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:16:58.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332117 2026-02-21T11:16:58.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332117 2026-02-21T11:16:58.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-42949672988 2-64424509467 3-296352743438 4-107374182426 5-322122547210 6-386547056646 7-158913789977 8-180388626455 9-412316860419 10-219043332117' 2026-02-21T11:16:58.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:58.180 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836509 2026-02-21T11:16:58.180 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:58.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:16:58.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836509 2026-02-21T11:16:58.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:58.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836509 2026-02-21T11:16:58.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-21T11:16:58.183 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 21474836509 2026-02-21T11:16:58.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:58.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836507 -lt 21474836509 2026-02-21T11:16:58.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:16:59.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:16:59.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:16:59.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836509 -lt 21474836509 2026-02-21T11:16:59.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:59.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672988 2026-02-21T11:16:59.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:59.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:16:59.601 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672988 2026-02-21T11:16:59.601 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:59.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672988 2026-02-21T11:16:59.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672988' 2026-02-21T11:16:59.603 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 42949672988 2026-02-21T11:16:59.603 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:16:59.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672988 -lt 42949672988 2026-02-21T11:16:59.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:16:59.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509467 2026-02-21T11:16:59.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:16:59.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:16:59.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509467 2026-02-21T11:16:59.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:16:59.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509467 2026-02-21T11:16:59.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509467' 2026-02-21T11:16:59.820 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 64424509467 2026-02-21T11:16:59.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:17:00.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509467 -lt 64424509467 2026-02-21T11:17:00.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:00.039 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743438 2026-02-21T11:17:00.039 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:00.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:17:00.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743438 2026-02-21T11:17:00.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:00.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743438 2026-02-21T11:17:00.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743438' 2026-02-21T11:17:00.042 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 296352743438 2026-02-21T11:17:00.042 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:17:00.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743438 -lt 296352743438 2026-02-21T11:17:00.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:00.251 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182426 2026-02-21T11:17:00.251 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:00.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:17:00.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182426 2026-02-21T11:17:00.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:00.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182426 2026-02-21T11:17:00.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182426' 2026-02-21T11:17:00.254 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.4 seq 107374182426 2026-02-21T11:17:00.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:17:00.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182426 -lt 107374182426 2026-02-21T11:17:00.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:00.466 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-322122547210 2026-02-21T11:17:00.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:00.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:17:00.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-322122547210 2026-02-21T11:17:00.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:00.469 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=322122547210 2026-02-21T11:17:00.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 322122547210' 2026-02-21T11:17:00.470 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.5 seq 322122547210 2026-02-21T11:17:00.470 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:17:00.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 322122547211 -lt 322122547210 2026-02-21T11:17:00.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:00.679 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-386547056646 2026-02-21T11:17:00.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:00.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:17:00.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-386547056646 2026-02-21T11:17:00.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:00.682 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=386547056646 2026-02-21T11:17:00.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 386547056646' 2026-02-21T11:17:00.683 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.6 seq 386547056646 2026-02-21T11:17:00.683 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:17:00.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 386547056647 -lt 386547056646 2026-02-21T11:17:00.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:00.891 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789977 2026-02-21T11:17:00.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:00.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:17:00.893 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789977 2026-02-21T11:17:00.893 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:00.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789977 2026-02-21T11:17:00.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789977' 2026-02-21T11:17:00.895 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.7 seq 158913789977 2026-02-21T11:17:00.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:17:01.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789977 -lt 158913789977 2026-02-21T11:17:01.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:01.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626455 2026-02-21T11:17:01.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:01.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:17:01.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626455 2026-02-21T11:17:01.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:01.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626455 2026-02-21T11:17:01.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626455' 2026-02-21T11:17:01.116 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.8 seq 180388626455 2026-02-21T11:17:01.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:17:01.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626455 -lt 180388626455 2026-02-21T11:17:01.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:01.332 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-412316860419 2026-02-21T11:17:01.332 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:01.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:17:01.333 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-412316860419 2026-02-21T11:17:01.333 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:01.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=412316860419 2026-02-21T11:17:01.335 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 412316860419' 2026-02-21T11:17:01.335 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.9 seq 412316860419 2026-02-21T11:17:01.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:17:01.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-21T11:17:01.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:01.554 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332117 2026-02-21T11:17:01.555 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:01.555 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:17:01.556 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332117 2026-02-21T11:17:01.556 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:01.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332117 2026-02-21T11:17:01.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332117' 2026-02-21T11:17:01.557 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.10 seq 219043332117 2026-02-21T11:17:01.558 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:17:01.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332118 -lt 219043332117 2026-02-21T11:17:01.786 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:17:01.786 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:01.787 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:02.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:02.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:02.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-21T11:17:02.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:02.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:02.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:02.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-21T11:17:02.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:17:02.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:17:02.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:338: TEST_chunk_mapping: delete_pool remap-pool 2026-02-21T11:17:02.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-21T11:17:02.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-21T11:17:02.874 INFO:tasks.workunit.client.0.trial166.stderr:pool 'remap-pool' does not exist 2026-02-21T11:17:02.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:339: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-21T11:17:03.155 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile remap-profile does not exist 2026-02-21T11:17:03.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:17:03.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-21T11:17:03.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-21T11:17:03.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: erasure_code_plugin_exists: local plugin=isa 2026-02-21T11:17:03.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: erasure_code_plugin_exists: local status 2026-02-21T11:17:03.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: erasure_code_plugin_exists: local grepstr 2026-02-21T11:17:03.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: erasure_code_plugin_exists: local s 2026-02-21T11:17:03.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: erasure_code_plugin_exists: case `uname` in 2026-02-21T11:17:03.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: erasure_code_plugin_exists: uname 2026-02-21T11:17:03.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2139: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-21T11:17:03.168 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-21T11:17:03.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: erasure_code_plugin_exists: s= 2026-02-21T11:17:03.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: erasure_code_plugin_exists: local status=0 2026-02-21T11:17:03.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-21T11:17:03.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-21T11:17:03.700 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-21T11:17:03.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2154: erasure_code_plugin_exists: return 0 2026-02-21T11:17:03.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-21T11:17:03.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-21T11:17:03.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-21T11:17:03.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-21T11:17:03.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-21T11:17:04.348 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-isa' already exists 2026-02-21T11:17:04.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:17:05.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-21T11:17:05.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:05.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-21T11:17:05.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:05.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-21T11:17:05.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:17:05.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:05.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:05.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-21T11:17:05.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-21T11:17:05.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-21T11:17:05.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:05.413 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-21T11:17:05.413 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:05.631 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-21T11:17:05.631 INFO:tasks.workunit.client.0.trial166.stderr:10 2026-02-21T11:17:05.631 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:17:05.631 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:17:05.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-21T11:17:05.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-21T11:17:05.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-21T11:17:05.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-21T11:17:05.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-21T11:17:05.897 INFO:tasks.workunit.client.0.trial166.stderr:osd.7 is already out. 2026-02-21T11:17:05.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-21T11:17:10.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-21T11:17:10.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-21T11:17:10.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:10.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-21T11:17:10.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-21T11:17:10.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:11.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-21T11:17:11.124 INFO:tasks.workunit.client.0.trial166.stderr:10 2026-02-21T11:17:11.124 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:17:11.124 INFO:tasks.workunit.client.0.trial166.stderr:1' 2026-02-21T11:17:11.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-21T11:17:11.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:11.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:11.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-21T11:17:11.373 INFO:tasks.workunit.client.0.trial166.stderr:osd.7 is already in. 2026-02-21T11:17:11.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:17:11.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-21T11:17:11.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-21T11:17:11.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-21T11:17:11.674 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-isa' does not exist 2026-02-21T11:17:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:17:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-21T11:17:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-21T11:17:11.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-21T11:17:11.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:17:11.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:11.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:11.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-21T11:17:11.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-21T11:17:11.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-21T11:17:11.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:11.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-21T11:17:11.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:11.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:9' 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-21T11:17:11.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-21T11:17:12.194 INFO:tasks.workunit.client.0.trial166.stderr:osd.9 is already out. 2026-02-21T11:17:12.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-21T11:17:17.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-21T11:17:17.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-21T11:17:17.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:17.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-21T11:17:17.207 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-21T11:17:17.207 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:17.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-21T11:17:17.419 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:17:17.419 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:17.419 INFO:tasks.workunit.client.0.trial166.stderr:5' 2026-02-21T11:17:17.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-21T11:17:17.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:17.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:17.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-21T11:17:17.668 INFO:tasks.workunit.client.0.trial166.stderr:osd.9 is already in. 2026-02-21T11:17:17.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:17:17.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-21T11:17:17.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-21T11:17:17.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-21T11:17:17.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-21T11:17:17.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-21T11:17:18.351 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:17:18.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:17:19.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-21T11:17:19.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:19.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-21T11:17:19.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:19.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-21T11:17:19.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:17:19.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:19.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:19.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-21T11:17:19.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-21T11:17:19.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:17:19.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:19.411 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-21T11:17:19.412 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:19.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-21T11:17:19.624 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:19.624 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-21T11:17:19.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-21T11:17:19.882 INFO:tasks.workunit.client.0.trial166.stderr:osd.10 is already out. 2026-02-21T11:17:19.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-21T11:17:24.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-21T11:17:24.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:17:24.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:24.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-21T11:17:24.894 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-21T11:17:24.894 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:25.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-21T11:17:25.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:25.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:25.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-21T11:17:25.354 INFO:tasks.workunit.client.0.trial166.stderr:osd.10 is already in. 2026-02-21T11:17:25.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:17:25.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-21T11:17:25.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-21T11:17:25.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:17:25.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:25.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:25.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-21T11:17:25.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-21T11:17:25.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:17:25.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:17:25.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:17:25.666 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:17:25.667 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:17:25.667 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:17:25.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:17:25.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:17:25.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:17:25.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:17:25.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:17:25.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:17:25.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:17:25.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:17:25.776 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:17:25.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:25.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:17:26.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836517 2026-02-21T11:17:26.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836517 2026-02-21T11:17:26.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517' 2026-02-21T11:17:26.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:17:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672996 2026-02-21T11:17:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672996 2026-02-21T11:17:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996' 2026-02-21T11:17:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:17:26.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509475 2026-02-21T11:17:26.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509475 2026-02-21T11:17:26.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475' 2026-02-21T11:17:26.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.230 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:17:26.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743446 2026-02-21T11:17:26.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743446 2026-02-21T11:17:26.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446' 2026-02-21T11:17:26.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:17:26.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182433 2026-02-21T11:17:26.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182433 2026-02-21T11:17:26.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433' 2026-02-21T11:17:26.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.386 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:17:26.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=322122547218 2026-02-21T11:17:26.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 322122547218 2026-02-21T11:17:26.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433 5-322122547218' 2026-02-21T11:17:26.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:17:26.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=386547056654 2026-02-21T11:17:26.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 386547056654 2026-02-21T11:17:26.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433 5-322122547218 6-386547056654' 2026-02-21T11:17:26.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.542 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:17:26.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789984 2026-02-21T11:17:26.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789984 2026-02-21T11:17:26.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433 5-322122547218 6-386547056654 7-158913789984' 2026-02-21T11:17:26.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:17:26.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626462 2026-02-21T11:17:26.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626462 2026-02-21T11:17:26.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433 5-322122547218 6-386547056654 7-158913789984 8-180388626462' 2026-02-21T11:17:26.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.702 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:17:26.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=412316860427 2026-02-21T11:17:26.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 412316860427 2026-02-21T11:17:26.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433 5-322122547218 6-386547056654 7-158913789984 8-180388626462 9-412316860427' 2026-02-21T11:17:26.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:26.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:17:26.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332125 2026-02-21T11:17:26.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332125 2026-02-21T11:17:26.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836517 1-42949672996 2-64424509475 3-296352743446 4-107374182433 5-322122547218 6-386547056654 7-158913789984 8-180388626462 9-412316860427 10-219043332125' 2026-02-21T11:17:26.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:26.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836517 2026-02-21T11:17:26.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:26.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:17:26.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836517 2026-02-21T11:17:26.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:26.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836517 2026-02-21T11:17:26.862 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836517 2026-02-21T11:17:26.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-21T11:17:26.862 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:17:27.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-21T11:17:27.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:27.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672996 2026-02-21T11:17:27.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:27.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:17:27.089 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672996 2026-02-21T11:17:27.089 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:27.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672996 2026-02-21T11:17:27.091 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 42949672996 2026-02-21T11:17:27.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672996' 2026-02-21T11:17:27.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:17:27.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672996 -lt 42949672996 2026-02-21T11:17:27.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:27.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509475 2026-02-21T11:17:27.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:27.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:17:27.308 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509475 2026-02-21T11:17:27.308 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:27.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509475 2026-02-21T11:17:27.310 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 64424509475 2026-02-21T11:17:27.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509475' 2026-02-21T11:17:27.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:17:27.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509475 -lt 64424509475 2026-02-21T11:17:27.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:27.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743446 2026-02-21T11:17:27.520 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:27.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:17:27.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743446 2026-02-21T11:17:27.521 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:27.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743446 2026-02-21T11:17:27.523 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 296352743446 2026-02-21T11:17:27.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743446' 2026-02-21T11:17:27.523 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:17:27.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743446 -lt 296352743446 2026-02-21T11:17:27.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:27.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182433 2026-02-21T11:17:27.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:27.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:17:27.743 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182433 2026-02-21T11:17:27.743 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:27.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182433 2026-02-21T11:17:27.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182433' 2026-02-21T11:17:27.745 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 107374182433 2026-02-21T11:17:27.745 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:17:27.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182433 -lt 107374182433 2026-02-21T11:17:27.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:27.970 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-322122547218 2026-02-21T11:17:27.970 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:27.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:17:27.972 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-322122547218 2026-02-21T11:17:27.972 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:27.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=322122547218 2026-02-21T11:17:27.973 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 322122547218 2026-02-21T11:17:27.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 322122547218' 2026-02-21T11:17:27.974 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:17:28.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 322122547218 -lt 322122547218 2026-02-21T11:17:28.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:28.193 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-386547056654 2026-02-21T11:17:28.193 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:28.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:17:28.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-386547056654 2026-02-21T11:17:28.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:28.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=386547056654 2026-02-21T11:17:28.196 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 386547056654 2026-02-21T11:17:28.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 386547056654' 2026-02-21T11:17:28.197 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:17:28.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 386547056654 -lt 386547056654 2026-02-21T11:17:28.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:28.420 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789984 2026-02-21T11:17:28.420 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:28.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:17:28.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789984 2026-02-21T11:17:28.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:28.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789984 2026-02-21T11:17:28.424 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.7 seq 158913789984 2026-02-21T11:17:28.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789984' 2026-02-21T11:17:28.424 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:17:28.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789984 -lt 158913789984 2026-02-21T11:17:28.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:28.648 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626462 2026-02-21T11:17:28.648 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:28.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:17:28.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626462 2026-02-21T11:17:28.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:28.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626462 2026-02-21T11:17:28.652 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 180388626462 2026-02-21T11:17:28.652 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626462' 2026-02-21T11:17:28.652 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:17:28.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626463 -lt 180388626462 2026-02-21T11:17:28.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:28.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-412316860427 2026-02-21T11:17:28.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:28.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:17:28.872 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-412316860427 2026-02-21T11:17:28.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:28.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=412316860427 2026-02-21T11:17:28.874 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.9 seq 412316860427 2026-02-21T11:17:28.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 412316860427' 2026-02-21T11:17:28.875 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:17:29.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 412316860427 -lt 412316860427 2026-02-21T11:17:29.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:29.100 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332125 2026-02-21T11:17:29.100 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:29.101 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:17:29.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332125 2026-02-21T11:17:29.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:29.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332125 2026-02-21T11:17:29.103 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.10 seq 219043332125 2026-02-21T11:17:29.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332125' 2026-02-21T11:17:29.104 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:17:29.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332125 -lt 219043332125 2026-02-21T11:17:29.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:17:29.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:29.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:29.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-21T11:17:29.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:29.596 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:29.596 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:29.597 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:29.597 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:29.597 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:29.597 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:29.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-21T11:17:29.802 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:29.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:29.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:30.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-21T11:17:30.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:17:30.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:17:30.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-21T11:17:30.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:17:30.078 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:30.078 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-21T11:17:30.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:30.300 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-21T11:17:30.300 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:30.300 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:30.300 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:30.300 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:30.300 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:30.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-21T11:17:30.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-21T11:17:30.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-21T11:17:30.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-21T11:17:30.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-21T11:17:30.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-21T11:17:30.545 INFO:tasks.workunit.client.0.trial166.stderr:osd.5 is already out. 2026-02-21T11:17:30.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-21T11:17:30.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-21T11:17:30.854 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 is already out. 2026-02-21T11:17:30.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-21T11:17:30.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:17:30.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:17:30.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:17:30.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:17:30.867 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:17:30.867 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:17:30.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:17:30.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:17:30.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:17:30.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:17:30.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:17:30.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:17:30.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:17:30.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:17:30.975 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:17:31.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:17:31.184 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:31.184 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:17:31.184 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:31.184 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:31.184 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.185 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:17:31.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836520 2026-02-21T11:17:31.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836520 2026-02-21T11:17:31.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520' 2026-02-21T11:17:31.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.265 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:17:31.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672999 2026-02-21T11:17:31.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672999 2026-02-21T11:17:31.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999' 2026-02-21T11:17:31.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:17:31.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509478 2026-02-21T11:17:31.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509478 2026-02-21T11:17:31.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478' 2026-02-21T11:17:31.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:17:31.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743449 2026-02-21T11:17:31.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743449 2026-02-21T11:17:31.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449' 2026-02-21T11:17:31.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.500 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:17:31.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182436 2026-02-21T11:17:31.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182436 2026-02-21T11:17:31.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436' 2026-02-21T11:17:31.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:17:31.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=322122547221 2026-02-21T11:17:31.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 322122547221 2026-02-21T11:17:31.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436 5-322122547221' 2026-02-21T11:17:31.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:17:31.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=386547056657 2026-02-21T11:17:31.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 386547056657 2026-02-21T11:17:31.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436 5-322122547221 6-386547056657' 2026-02-21T11:17:31.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.735 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:17:31.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789987 2026-02-21T11:17:31.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789987 2026-02-21T11:17:31.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436 5-322122547221 6-386547056657 7-158913789987' 2026-02-21T11:17:31.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:17:31.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626466 2026-02-21T11:17:31.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626466 2026-02-21T11:17:31.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436 5-322122547221 6-386547056657 7-158913789987 8-180388626466' 2026-02-21T11:17:31.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:17:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=412316860430 2026-02-21T11:17:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 412316860430 2026-02-21T11:17:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436 5-322122547221 6-386547056657 7-158913789987 8-180388626466 9-412316860430' 2026-02-21T11:17:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:31.976 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:17:32.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332128 2026-02-21T11:17:32.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332128 2026-02-21T11:17:32.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-42949672999 2-64424509478 3-296352743449 4-107374182436 5-322122547221 6-386547056657 7-158913789987 8-180388626466 9-412316860430 10-219043332128' 2026-02-21T11:17:32.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:32.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836520 2026-02-21T11:17:32.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:32.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:17:32.057 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836520 2026-02-21T11:17:32.057 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:32.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836520 2026-02-21T11:17:32.059 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836520 2026-02-21T11:17:32.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836520' 2026-02-21T11:17:32.059 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:17:32.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836517 -lt 21474836520 2026-02-21T11:17:32.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:17:33.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:17:33.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:17:33.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836520 -lt 21474836520 2026-02-21T11:17:33.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:33.489 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672999 2026-02-21T11:17:33.489 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:33.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:17:33.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672999 2026-02-21T11:17:33.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:33.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672999 2026-02-21T11:17:33.492 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 42949672999 2026-02-21T11:17:33.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672999' 2026-02-21T11:17:33.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:17:33.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672999 -lt 42949672999 2026-02-21T11:17:33.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:33.705 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509478 2026-02-21T11:17:33.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:33.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:17:33.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509478 2026-02-21T11:17:33.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:33.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509478 2026-02-21T11:17:33.708 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 64424509478 2026-02-21T11:17:33.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509478' 2026-02-21T11:17:33.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:17:33.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509478 -lt 64424509478 2026-02-21T11:17:33.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:33.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743449 2026-02-21T11:17:33.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:33.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:17:33.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743449 2026-02-21T11:17:33.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:33.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743449 2026-02-21T11:17:33.929 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 296352743449 2026-02-21T11:17:33.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743449' 2026-02-21T11:17:33.929 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:17:34.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743449 -lt 296352743449 2026-02-21T11:17:34.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:34.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182436 2026-02-21T11:17:34.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:34.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:17:34.153 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182436 2026-02-21T11:17:34.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:34.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182436 2026-02-21T11:17:34.155 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 107374182436 2026-02-21T11:17:34.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182436' 2026-02-21T11:17:34.155 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:17:34.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182437 -lt 107374182436 2026-02-21T11:17:34.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:34.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-322122547221 2026-02-21T11:17:34.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:34.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:17:34.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-322122547221 2026-02-21T11:17:34.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=322122547221 2026-02-21T11:17:34.369 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 322122547221 2026-02-21T11:17:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 322122547221' 2026-02-21T11:17:34.369 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:17:34.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 322122547221 -lt 322122547221 2026-02-21T11:17:34.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:34.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-386547056657 2026-02-21T11:17:34.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:34.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:17:34.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-386547056657 2026-02-21T11:17:34.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:34.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=386547056657 2026-02-21T11:17:34.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 386547056657' 2026-02-21T11:17:34.593 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 386547056657 2026-02-21T11:17:34.594 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:17:34.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 386547056658 -lt 386547056657 2026-02-21T11:17:34.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:34.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789987 2026-02-21T11:17:34.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:34.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:17:34.823 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789987 2026-02-21T11:17:34.823 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:34.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789987 2026-02-21T11:17:34.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789987' 2026-02-21T11:17:34.824 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.7 seq 158913789987 2026-02-21T11:17:34.825 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:17:35.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789988 -lt 158913789987 2026-02-21T11:17:35.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:35.050 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626466 2026-02-21T11:17:35.051 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:35.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:17:35.053 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626466 2026-02-21T11:17:35.053 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:35.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626466 2026-02-21T11:17:35.054 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 180388626466 2026-02-21T11:17:35.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626466' 2026-02-21T11:17:35.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:17:35.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626466 -lt 180388626466 2026-02-21T11:17:35.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:35.267 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-412316860430 2026-02-21T11:17:35.267 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:35.268 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:17:35.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-412316860430 2026-02-21T11:17:35.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:35.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=412316860430 2026-02-21T11:17:35.271 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.9 seq 412316860430 2026-02-21T11:17:35.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 412316860430' 2026-02-21T11:17:35.271 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:17:35.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 412316860430 -lt 412316860430 2026-02-21T11:17:35.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:35.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332128 2026-02-21T11:17:35.490 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:35.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:17:35.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332128 2026-02-21T11:17:35.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:35.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332128 2026-02-21T11:17:35.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332128' 2026-02-21T11:17:35.494 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.10 seq 219043332128 2026-02-21T11:17:35.494 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:17:35.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332129 -lt 219043332128 2026-02-21T11:17:35.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:17:35.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:35.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:36.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:36.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:36.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=26 2026-02-21T11:17:36.233 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:36.233 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:36.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:36.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 26 = 28 2026-02-21T11:17:36.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 26 '!=' -1 2026-02-21T11:17:36.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T11:17:36.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=26 2026-02-21T11:17:36.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T11:17:36.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T11:17:36.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T11:17:36.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:36.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:36.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:36.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:36.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:36.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:36.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:36.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=27 2026-02-21T11:17:36.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:36.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:36.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:37.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 27 = 28 2026-02-21T11:17:37.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 27 '!=' 26 2026-02-21T11:17:37.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T11:17:37.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=27 2026-02-21T11:17:37.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T11:17:37.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T11:17:37.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T11:17:37.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:37.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:37.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:37.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:37.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:37.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:37.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:37.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=27 2026-02-21T11:17:37.447 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:37.448 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:37.448 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:37.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 27 = 28 2026-02-21T11:17:37.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 27 '!=' 27 2026-02-21T11:17:37.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T11:17:37.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T11:17:37.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T11:17:37.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T11:17:37.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T11:17:37.726 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T11:17:37.726 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T11:17:38.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=2688 2026-02-21T11:17:38.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test 2688 '!=' null 2026-02-21T11:17:38.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1668: wait_for_clean: loop=0 2026-02-21T11:17:38.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T11:17:38.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T11:17:38.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T11:17:38.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:38.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:38.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:38.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:38.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:38.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:38.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:38.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=27 2026-02-21T11:17:38.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:38.322 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:38.322 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:38.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 27 = 28 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 27 '!=' 27 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T11:17:38.613 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T11:17:38.614 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T11:17:38.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=1023 2026-02-21T11:17:38.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test 1023 '!=' null 2026-02-21T11:17:38.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1668: wait_for_clean: loop=0 2026-02-21T11:17:38.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T11:17:38.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T11:17:38.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T11:17:38.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:38.994 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:38.994 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:38.994 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:38.994 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:38.994 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:38.995 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:39.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-21T11:17:39.203 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:39.203 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:39.203 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:17:39.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-21T11:17:39.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:39.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-21T11:17:39.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:39.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-21T11:17:39.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-21T11:17:39.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:39.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:39.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-21T11:17:39.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-21T11:17:40.216 INFO:tasks.workunit.client.0.trial166.stderr:osd.5 is already in. 2026-02-21T11:17:40.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-21T11:17:40.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-21T11:17:40.496 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 is already in. 2026-02-21T11:17:40.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-21T11:17:40.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:17:40.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:17:40.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:17:40.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:17:40.509 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:17:40.509 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:17:40.509 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:17:40.509 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:17:40.509 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:17:40.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:17:40.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:17:40.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:17:40.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:17:40.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:17:40.614 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:17:40.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:17:40.833 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:8 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:17:40.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:40.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:17:40.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836524 2026-02-21T11:17:40.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836524 2026-02-21T11:17:40.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524' 2026-02-21T11:17:40.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:40.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:17:40.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949673003 2026-02-21T11:17:40.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949673003 2026-02-21T11:17:40.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003' 2026-02-21T11:17:40.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:40.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:17:41.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509482 2026-02-21T11:17:41.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509482 2026-02-21T11:17:41.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482' 2026-02-21T11:17:41.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:17:41.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=296352743453 2026-02-21T11:17:41.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 296352743453 2026-02-21T11:17:41.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453' 2026-02-21T11:17:41.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:17:41.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182440 2026-02-21T11:17:41.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182440 2026-02-21T11:17:41.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440' 2026-02-21T11:17:41.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.228 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:17:41.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=322122547225 2026-02-21T11:17:41.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 322122547225 2026-02-21T11:17:41.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440 5-322122547225' 2026-02-21T11:17:41.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:17:41.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=386547056661 2026-02-21T11:17:41.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 386547056661 2026-02-21T11:17:41.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440 5-322122547225 6-386547056661' 2026-02-21T11:17:41.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:17:41.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789991 2026-02-21T11:17:41.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789991 2026-02-21T11:17:41.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440 5-322122547225 6-386547056661 7-158913789991' 2026-02-21T11:17:41.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:17:41.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626469 2026-02-21T11:17:41.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626469 2026-02-21T11:17:41.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440 5-322122547225 6-386547056661 7-158913789991 8-180388626469' 2026-02-21T11:17:41.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.549 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-21T11:17:41.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=412316860434 2026-02-21T11:17:41.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 412316860434 2026-02-21T11:17:41.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440 5-322122547225 6-386547056661 7-158913789991 8-180388626469 9-412316860434' 2026-02-21T11:17:41.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:17:41.627 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-21T11:17:41.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332132 2026-02-21T11:17:41.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332132 2026-02-21T11:17:41.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836524 1-42949673003 2-64424509482 3-296352743453 4-107374182440 5-322122547225 6-386547056661 7-158913789991 8-180388626469 9-412316860434 10-219043332132' 2026-02-21T11:17:41.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:41.710 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836524 2026-02-21T11:17:41.710 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:41.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:17:41.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836524 2026-02-21T11:17:41.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:41.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836524 2026-02-21T11:17:41.713 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 21474836524 2026-02-21T11:17:41.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836524' 2026-02-21T11:17:41.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:17:41.936 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836521 -lt 21474836524 2026-02-21T11:17:41.936 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:17:42.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:17:42.938 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:17:43.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836524 -lt 21474836524 2026-02-21T11:17:43.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:43.146 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949673003 2026-02-21T11:17:43.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:43.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:17:43.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949673003 2026-02-21T11:17:43.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:43.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949673003 2026-02-21T11:17:43.150 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 42949673003 2026-02-21T11:17:43.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949673003' 2026-02-21T11:17:43.151 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:17:43.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949673003 -lt 42949673003 2026-02-21T11:17:43.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:43.377 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509482 2026-02-21T11:17:43.377 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:43.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:17:43.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509482 2026-02-21T11:17:43.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:43.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509482 2026-02-21T11:17:43.381 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 64424509482 2026-02-21T11:17:43.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509482' 2026-02-21T11:17:43.381 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:17:43.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509482 -lt 64424509482 2026-02-21T11:17:43.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:43.603 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-296352743453 2026-02-21T11:17:43.603 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:43.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:17:43.605 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-296352743453 2026-02-21T11:17:43.605 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:43.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=296352743453 2026-02-21T11:17:43.607 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 296352743453 2026-02-21T11:17:43.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 296352743453' 2026-02-21T11:17:43.607 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:17:43.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 296352743453 -lt 296352743453 2026-02-21T11:17:43.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:43.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-107374182440 2026-02-21T11:17:43.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:43.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:17:43.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-107374182440 2026-02-21T11:17:43.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:43.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182440 2026-02-21T11:17:43.821 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 107374182440 2026-02-21T11:17:43.821 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 107374182440' 2026-02-21T11:17:43.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:17:44.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182441 -lt 107374182440 2026-02-21T11:17:44.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:44.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-322122547225 2026-02-21T11:17:44.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:44.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:17:44.047 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-322122547225 2026-02-21T11:17:44.047 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:44.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=322122547225 2026-02-21T11:17:44.049 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 322122547225' 2026-02-21T11:17:44.049 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 322122547225 2026-02-21T11:17:44.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:17:44.269 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 322122547225 -lt 322122547225 2026-02-21T11:17:44.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:44.270 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-386547056661 2026-02-21T11:17:44.271 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:44.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:17:44.272 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-386547056661 2026-02-21T11:17:44.272 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:44.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=386547056661 2026-02-21T11:17:44.274 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 386547056661 2026-02-21T11:17:44.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 386547056661' 2026-02-21T11:17:44.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:17:44.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 386547056661 -lt 386547056661 2026-02-21T11:17:44.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:44.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-158913789991 2026-02-21T11:17:44.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:44.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:17:44.497 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-158913789991 2026-02-21T11:17:44.497 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:44.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789991 2026-02-21T11:17:44.499 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.7 seq 158913789991 2026-02-21T11:17:44.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 158913789991' 2026-02-21T11:17:44.499 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:17:44.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789992 -lt 158913789991 2026-02-21T11:17:44.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:44.719 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-180388626469 2026-02-21T11:17:44.719 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:44.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:17:44.721 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-180388626469 2026-02-21T11:17:44.721 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:44.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626469 2026-02-21T11:17:44.722 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 180388626469 2026-02-21T11:17:44.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 180388626469' 2026-02-21T11:17:44.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:17:44.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626470 -lt 180388626469 2026-02-21T11:17:44.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:44.954 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-412316860434 2026-02-21T11:17:44.954 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:44.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-21T11:17:44.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-412316860434 2026-02-21T11:17:44.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:44.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=412316860434 2026-02-21T11:17:44.958 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.9 seq 412316860434 2026-02-21T11:17:44.958 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 412316860434' 2026-02-21T11:17:44.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-21T11:17:45.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 412316860434 -lt 412316860434 2026-02-21T11:17:45.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:17:45.170 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-219043332132 2026-02-21T11:17:45.170 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:17:45.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-21T11:17:45.172 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-219043332132 2026-02-21T11:17:45.173 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:17:45.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332132 2026-02-21T11:17:45.174 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.10 seq 219043332132 2026-02-21T11:17:45.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 219043332132' 2026-02-21T11:17:45.174 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-21T11:17:45.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332133 -lt 219043332132 2026-02-21T11:17:45.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:17:45.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:45.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:45.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-21T11:17:45.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:17:45.674 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:17:45.674 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:17:45.674 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:17:45.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:17:45.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:17:45.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:17:45.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-21T11:17:45.883 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:17:45.884 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:17:45.884 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:17:46.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-21T11:17:46.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:17:46.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:17:46.155 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-21T11:17:46.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:17:46.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:46.156 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-21T11:17:46.156 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:46.364 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-21T11:17:46.364 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:17:46.364 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:46.364 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:17:46.364 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:46.365 INFO:tasks.workunit.client.0.trial166.stderr:10' 2026-02-21T11:17:46.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-21T11:17:46.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-21T11:17:46.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:17:46.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-21T11:17:46.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-21T11:17:46.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:17:46.668 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:17:46.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-21T11:17:46.945 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-21T11:17:46.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:17:46.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-21T11:17:46.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-21T11:17:46.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-21T11:17:46.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-21T11:17:46.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-21T11:17:47.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-21T11:17:47.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-21T11:17:47.602 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-lrc-a' already exists 2026-02-21T11:17:47.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:17:48.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-21T11:17:48.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-21T11:17:48.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-21T11:17:48.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-21T11:17:48.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:48.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-21T11:17:48.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:17:48.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:48.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:48.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-21T11:17:48.662 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-21T11:17:48.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-21T11:17:48.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:48.663 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-21T11:17:48.663 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:48.878 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-21T11:17:48.878 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:48.878 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:17:48.878 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-21T11:17:48.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-21T11:17:48.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-21T11:17:48.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-21T11:17:48.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-21T11:17:49.100 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 is already out. 2026-02-21T11:17:49.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-21T11:17:54.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-21T11:17:54.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-21T11:17:54.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:54.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-21T11:17:54.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-21T11:17:54.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:54.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-21T11:17:54.336 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:54.336 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:17:54.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-21T11:17:54.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:54.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:54.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-21T11:17:54.581 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 is already in. 2026-02-21T11:17:54.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:17:54.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-21T11:17:54.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-21T11:17:54.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-21T11:17:54.884 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-lrc-a' does not exist 2026-02-21T11:17:54.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-21T11:17:55.154 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-21T11:17:55.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:17:55.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-21T11:17:55.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-21T11:17:55.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-21T11:17:55.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-21T11:17:55.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-21T11:17:55.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-21T11:17:55.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-21T11:17:55.810 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-lrc' already exists 2026-02-21T11:17:55.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:17:56.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-21T11:17:56.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-21T11:17:56.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-21T11:17:56.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-21T11:17:56.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-21T11:17:56.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:17:56.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:17:56.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:17:56.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-21T11:17:56.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-21T11:17:56.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-21T11:17:56.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:17:56.872 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-21T11:17:56.872 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:10 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:17:57.089 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:17:57.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-21T11:17:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-21T11:17:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-21T11:17:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-21T11:17:57.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-21T11:17:57.350 INFO:tasks.workunit.client.0.trial166.stderr:osd.7 is already out. 2026-02-21T11:17:57.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-21T11:18:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-21T11:18:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-21T11:18:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:18:02.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-21T11:18:02.364 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-21T11:18:02.364 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:18:02.578 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:10 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:4' 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-21T11:18:02.579 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:18:02.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:18:02.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-21T11:18:02.829 INFO:tasks.workunit.client.0.trial166.stderr:osd.7 is already in. 2026-02-21T11:18:02.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:18:02.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-21T11:18:02.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-21T11:18:02.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-21T11:18:03.129 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-lrc' does not exist 2026-02-21T11:18:03.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-21T11:18:03.406 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile profile-lrc does not exist 2026-02-21T11:18:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-21T11:18:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-21T11:18:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-21T11:18:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-21T11:18:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-21T11:18:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-21T11:18:03.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-21T11:18:03.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-21T11:18:04.081 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-shec' already exists 2026-02-21T11:18:04.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:18:05.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-21T11:18:05.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-21T11:18:05.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-21T11:18:05.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-21T11:18:05.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-21T11:18:05.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:18:05.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:18:05.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-21T11:18:05.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-21T11:18:05.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-21T11:18:05.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:18:05.135 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-21T11:18:05.135 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:18:05.342 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-21T11:18:05.342 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:18:05.342 INFO:tasks.workunit.client.0.trial166.stderr:1' 2026-02-21T11:18:05.342 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-21T11:18:05.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-21T11:18:05.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-21T11:18:05.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-21T11:18:05.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-21T11:18:05.600 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 is already out. 2026-02-21T11:18:05.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-21T11:18:10.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-21T11:18:10.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-21T11:18:10.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-21T11:18:10.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-21T11:18:10.613 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-21T11:18:10.613 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:18:10.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-21T11:18:10.832 INFO:tasks.workunit.client.0.trial166.stderr:9 2026-02-21T11:18:10.832 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:18:10.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-21T11:18:10.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-21T11:18:10.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-21T11:18:10.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-21T11:18:11.138 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 is already in. 2026-02-21T11:18:11.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-21T11:18:11.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-21T11:18:11.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-21T11:18:11.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-21T11:18:11.406 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-shec' does not exist 2026-02-21T11:18:11.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-21T11:18:11.717 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile profile-shec does not exist 2026-02-21T11:18:11.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-21T11:18:11.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-21T11:18:11.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-21T11:18:11.996 INFO:tasks.workunit.client.0.trial166.stderr:pool 'ecpool' does not exist 2026-02-21T11:18:12.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-21T11:18:12.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:18:12.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:18:12.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:18:12.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:18:12.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:18:12.121 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:18:12.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:18:12.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:18:12.123 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:18:12.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:18:12.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:18:12.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:18:12.125 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:18:12.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:18:12.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:18:12.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-21T11:18:12.179 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:18:12.179 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.179 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:18:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46206 2026-02-21T11:18:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:18:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/test-erasure-code 0 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:18:12.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:18:12.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:18:12.184 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:18:12.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:18:12.185 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:18:12.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:18:12.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:18:12.187 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:18:12.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:18:12.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:18:12.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:18:12.189 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:18:12.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:18:12.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T11:18:12.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-21T11:18:12.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:18:12.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46206 2026-02-21T11:18:12.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46206 2026-02-21T11:18:12.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:18:12.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:18:12.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-21T11:18:12.194 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T11:18:12.194 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T11:18:12.243 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-21T11:18:12.243 DEBUG:teuthology.orchestra.run.trial166:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7273f3049146bdd7fa7b4a400114872a50228e9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-21T11:18:12.292 INFO:tasks.workunit.client.0.trial166.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-21T11:18:12.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/test-erasure-eio 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none ' 2026-02-21T11:18:12.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-21T11:18:12.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-21T11:18:12.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-21T11:18:12.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T11:18:12.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-21T11:18:12.300 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_object_attr_read_error 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_recovery_multiple_errors 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_recovery_multiple_objects 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_recovery_unfound 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_ec_single_recovery_error 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_bad_size_shard_0 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_bad_size_shard_1 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_subread_missing 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:18:12.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:18:12.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:18:12.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:18:12.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:18:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:18:12.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:18:12.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:18:12.305 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:18:12.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:18:12.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:18:12.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:18:12.307 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:18:12.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:18:12.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:18:12.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:18:12.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:18:12.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:12.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:18:12.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:18:12.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:18:12.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:18:12.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:18:12.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:12.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:18:12.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:18:12.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T11:18:12.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:18:12.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:18:12.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:18:12.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:12.344 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:12.344 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:12.344 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.344 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:12.344 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:12.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:18:12.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:18:12.373 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:18:12.373 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:18:12.374 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:18:12.374 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:18:12.374 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:18:12.374 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:18:12.374 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:18:12.374 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:18:12.375 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:12.375 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.375 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:12.375 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:18:12.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:18:12.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:18:12.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:18:12.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:18:12.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:18:12.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:18:12.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:18:12.429 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:18:12.430 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:18:12.430 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:18:12.430 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:12.431 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.431 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:12.431 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:18:12.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:18:12.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:18:12.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:18:12.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:18:12.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:12.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:12.593 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:12.593 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:12.593 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:12.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:12.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:18:12.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:18:12.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:18:12.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:18:12.719 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:18:12.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:18:13.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:18:13.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:18:13.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:18:13.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:13.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:13.731 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:13.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:18:13.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:18:13.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:18:13.779 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:18:13.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:18:13.787 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:18:12.356+0000 7f603fa67ec0 0 load: jerasure load: lrc 2026-02-21T11:18:13.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:18:13.788 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-21T11:18:13.789 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:18:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:13.791 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:18:13.792 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:18:13.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=89f0fcb7-d1fc-4a39-8b22-2cafb42547c8 2026-02-21T11:18:13.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 89f0fcb7-d1fc-4a39-8b22-2cafb42547c8' 2026-02-21T11:18:13.793 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 89f0fcb7-d1fc-4a39-8b22-2cafb42547c8 2026-02-21T11:18:13.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:18:13.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB1lJlpF3zdLxAAB/eXgG3U9Lc2kXxuv5aKQQ== 2026-02-21T11:18:13.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB1lJlpF3zdLxAAB/eXgG3U9Lc2kXxuv5aKQQ=="}' 2026-02-21T11:18:13.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 89f0fcb7-d1fc-4a39-8b22-2cafb42547c8 -i td/test-erasure-eio/0/new.json 2026-02-21T11:18:13.916 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:13.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:18:13.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB1lJlpF3zdLxAAB/eXgG3U9Lc2kXxuv5aKQQ== --osd-uuid 89f0fcb7-d1fc-4a39-8b22-2cafb42547c8 2026-02-21T11:18:13.940 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:13.936+0000 7f336cd22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:13.942 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:13.936+0000 7f336cd22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:13.943 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:13.936+0000 7f336cd22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:13.943 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:13.940+0000 7f336cd22a40 -1 bdev(0x55ba445e0400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:18:13.944 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:13.940+0000 7f336cd22a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:18:14.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:18:14.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:18:14.608 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:18:14.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:18:14.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:18:14.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:18:14.720 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:18:14.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:18:14.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:18:14.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:18:14.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:18:14.736 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:14.732+0000 7f8d72042a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:14.738 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:14.732+0000 7f8d72042a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:14.739 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:14.732+0000 7f8d72042a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:14.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:18:14.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:18:14.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:18:14.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:18:14.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:18:14.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:14.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:18:14.862 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:14.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:14.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:18:14.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:15.165 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:15.160+0000 7f8d72042a40 -1 Falling back to public interface 2026-02-21T11:18:15.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:15.764+0000 7f8d72042a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:18:15.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:15.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:15.976 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:15.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:18:15.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:15.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:18:16.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:17.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:17.202 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:18:17.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:17.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:18:17.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:17.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:18:17.419 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1297250877,v1:127.0.0.1:6803/1297250877] [v2:127.0.0.1:6804/1297250877,v1:127.0.0.1:6805/1297250877] exists,up 89f0fcb7-d1fc-4a39-8b22-2cafb42547c8 2026-02-21T11:18:17.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:18:17.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:18:17.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5b76373d-de00-49fb-9d51-e587db109105 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 5b76373d-de00-49fb-9d51-e587db109105 2026-02-21T11:18:17.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5b76373d-de00-49fb-9d51-e587db109105' 2026-02-21T11:18:17.425 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:18:17.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB5lJlpgL/vGRAAunMH918iWq4Mywo/o485dQ== 2026-02-21T11:18:17.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB5lJlpgL/vGRAAunMH918iWq4Mywo/o485dQ=="}' 2026-02-21T11:18:17.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5b76373d-de00-49fb-9d51-e587db109105 -i td/test-erasure-eio/1/new.json 2026-02-21T11:18:17.654 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:17.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:18:17.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB5lJlpgL/vGRAAunMH918iWq4Mywo/o485dQ== --osd-uuid 5b76373d-de00-49fb-9d51-e587db109105 2026-02-21T11:18:17.691 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:17.684+0000 7efe16be6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:17.693 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:17.688+0000 7efe16be6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:17.694 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:17.688+0000 7efe16be6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:17.695 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:17.688+0000 7efe16be6a40 -1 bdev(0x5569670e2400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:18:17.695 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:17.688+0000 7efe16be6a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:18:18.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:18:18.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:18:18.332 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:18:18.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:18:18.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:18:18.637 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:18:18.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:18:18.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:18:18.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:18:18.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:18:18.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:18:18.663 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:18.656+0000 7f20a223fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:18.665 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:18.660+0000 7f20a223fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:18.666 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:18.660+0000 7f20a223fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:18.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:18:18.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:18:18.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:18:18.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:18:18.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:18:18.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:18.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:18:18.847 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:18.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:18.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:18:19.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:19.117 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:19.112+0000 7f20a223fa40 -1 Falling back to public interface 2026-02-21T11:18:19.681 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:19.676+0000 7f20a223fa40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:18:20.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:20.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:20.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:18:20.069 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:20.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:20.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:18:20.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:21.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:21.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:21.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:18:21.287 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:18:21.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:21.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:18:21.500 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1251203527,v1:127.0.0.1:6811/1251203527] [v2:127.0.0.1:6812/1251203527,v1:127.0.0.1:6813/1251203527] exists,up 5b76373d-de00-49fb-9d51-e587db109105 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:18:21.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:18:21.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:18:21.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:18:21.507 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:18:21.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2a651e32-014f-4716-adb5-ba7a89280c5c 2026-02-21T11:18:21.508 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 2a651e32-014f-4716-adb5-ba7a89280c5c 2026-02-21T11:18:21.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2a651e32-014f-4716-adb5-ba7a89280c5c' 2026-02-21T11:18:21.509 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:18:21.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB9lJlpq03BHhAAiju1WIT2omGBFqaM3O+poQ== 2026-02-21T11:18:21.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB9lJlpq03BHhAAiju1WIT2omGBFqaM3O+poQ=="}' 2026-02-21T11:18:21.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2a651e32-014f-4716-adb5-ba7a89280c5c -i td/test-erasure-eio/2/new.json 2026-02-21T11:18:21.736 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:18:21.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:18:21.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB9lJlpq03BHhAAiju1WIT2omGBFqaM3O+poQ== --osd-uuid 2a651e32-014f-4716-adb5-ba7a89280c5c 2026-02-21T11:18:21.766 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:21.760+0000 7ff0372b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:21.768 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:21.764+0000 7ff0372b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:21.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:21.764+0000 7ff0372b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:21.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:21.764+0000 7ff0372b5a40 -1 bdev(0x558bd66f6400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:18:21.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:21.764+0000 7ff0372b5a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:18:22.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:18:22.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:18:22.383 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:18:22.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:18:22.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:18:22.676 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:18:22.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:18:22.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:18:22.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:18:22.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:18:22.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:18:22.694 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:22.688+0000 7fa8dd102a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:22.696 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:22.692+0000 7fa8dd102a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:22.697 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:22.692+0000 7fa8dd102a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:22.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:18:22.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:18:22.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:18:22.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:18:22.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:18:22.898 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:22.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:22.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:18:22.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:22.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:18:23.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:23.169 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:23.164+0000 7fa8dd102a40 -1 Falling back to public interface 2026-02-21T11:18:23.721 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:23.716+0000 7fa8dd102a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:18:24.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:24.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:24.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:18:24.112 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:24.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:24.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:18:24.337 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3819687528,v1:127.0.0.1:6819/3819687528] [v2:127.0.0.1:6820/3819687528,v1:127.0.0.1:6821/3819687528] exists,up 2a651e32-014f-4716-adb5-ba7a89280c5c 2026-02-21T11:18:24.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:18:24.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:18:24.342 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:18:24.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:18:24.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:18:24.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:18:24.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:18:24.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:18:24.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:18:24.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:18:24.345 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:18:24.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7a88f890-c586-48e1-9653-a0eedb08b72d 2026-02-21T11:18:24.345 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 7a88f890-c586-48e1-9653-a0eedb08b72d 2026-02-21T11:18:24.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 7a88f890-c586-48e1-9653-a0eedb08b72d' 2026-02-21T11:18:24.346 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:18:24.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCAlJlphVENFRAA5hXqFLeSV4T7BLCjme53aA== 2026-02-21T11:18:24.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCAlJlphVENFRAA5hXqFLeSV4T7BLCjme53aA=="}' 2026-02-21T11:18:24.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7a88f890-c586-48e1-9653-a0eedb08b72d -i td/test-erasure-eio/3/new.json 2026-02-21T11:18:24.570 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:18:24.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:18:24.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAlJlphVENFRAA5hXqFLeSV4T7BLCjme53aA== --osd-uuid 7a88f890-c586-48e1-9653-a0eedb08b72d 2026-02-21T11:18:24.599 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:24.592+0000 7efed09aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:24.600 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:24.596+0000 7efed09aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:24.601 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:24.596+0000 7efed09aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:24.602 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:24.596+0000 7efed09aba40 -1 bdev(0x55af76df4400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:18:24.602 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:24.596+0000 7efed09aba40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:18:25.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:18:25.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:18:25.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:18:25.301 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:18:25.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:18:25.582 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:18:25.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:18:25.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:18:25.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:18:25.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:18:25.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:18:25.609 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:25.604+0000 7ff81bbaba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:25.611 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:25.604+0000 7ff81bbaba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:25.612 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:25.608+0000 7ff81bbaba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:25.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:18:25.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:18:25.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:18:25.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:18:25.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:18:25.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:25.795 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:25.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:18:25.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:25.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:18:25.925 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:25.920+0000 7ff81bbaba40 -1 Falling back to public interface 2026-02-21T11:18:26.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:26.597 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:26.592+0000 7ff81bbaba40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:18:27.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:27.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:27.011 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:27.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:18:27.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:27.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:18:27.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:28.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:28.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:28.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:18:28.233 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:18:28.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:28.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:18:28.441 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2136081638,v1:127.0.0.1:6827/2136081638] [v2:127.0.0.1:6828/2136081638,v1:127.0.0.1:6829/2136081638] exists,up 7a88f890-c586-48e1-9653-a0eedb08b72d 2026-02-21T11:18:28.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:18:28.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:18:28.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:28.447 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:18:28.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:18:28.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:18:28.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:18:28.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:18:28.449 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:18:28.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6a48d891-a088-4fc3-a561-61a6fdd60609 2026-02-21T11:18:28.450 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 6a48d891-a088-4fc3-a561-61a6fdd60609 2026-02-21T11:18:28.450 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 6a48d891-a088-4fc3-a561-61a6fdd60609' 2026-02-21T11:18:28.450 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:18:28.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCElJlpZC5BGxAA1RruPC9C7SQKGoEiEsYYew== 2026-02-21T11:18:28.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCElJlpZC5BGxAA1RruPC9C7SQKGoEiEsYYew=="}' 2026-02-21T11:18:28.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6a48d891-a088-4fc3-a561-61a6fdd60609 -i td/test-erasure-eio/4/new.json 2026-02-21T11:18:28.669 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:18:28.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:18:28.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCElJlpZC5BGxAA1RruPC9C7SQKGoEiEsYYew== --osd-uuid 6a48d891-a088-4fc3-a561-61a6fdd60609 2026-02-21T11:18:28.698 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:28.692+0000 7fab1efa1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:28.699 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:28.692+0000 7fab1efa1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:28.700 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:28.696+0000 7fab1efa1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:28.701 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:28.696+0000 7fab1efa1a40 -1 bdev(0x557c0a648400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:18:28.701 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:28.696+0000 7fab1efa1a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:18:29.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:18:29.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:18:29.359 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:18:29.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:18:29.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:18:29.647 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:18:29.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:18:29.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:18:29.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:18:29.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:18:29.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:18:29.674 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:29.668+0000 7fc996732a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:29.676 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:29.672+0000 7fc996732a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:29.677 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:29.672+0000 7fc996732a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:29.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:18:29.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:18:29.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:18:29.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:18:29.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:18:29.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:29.862 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:29.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:18:29.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:29.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:18:30.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:30.117 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:30.112+0000 7fc996732a40 -1 Falling back to public interface 2026-02-21T11:18:30.725 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:30.720+0000 7fc996732a40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:18:31.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:31.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:31.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:18:31.079 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:31.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:31.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:18:31.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:32.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:32.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:32.315 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:18:32.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:18:32.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:32.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:18:32.529 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3055201320,v1:127.0.0.1:6835/3055201320] [v2:127.0.0.1:6836/3055201320,v1:127.0.0.1:6837/3055201320] exists,up 6a48d891-a088-4fc3-a561-61a6fdd60609 2026-02-21T11:18:32.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:18:32.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:18:32.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:18:32.531 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:18:32.531 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:18:32.531 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:18:32.532 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:32.532 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:32.532 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:32.532 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:18:32.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:18:32.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:18:32.594 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:18:32.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:18:32.602 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:18:15.164+0000 7f8d72042a40 0 load: jerasure load: lrc 2026-02-21T11:18:32.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:18:32.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-21T11:18:32.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-21T11:18:32.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:18:32.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:18:32.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-21T11:18:32.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:18:32.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-21T11:18:32.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:18:32.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-21T11:18:32.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:18:32.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:18:33.252 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:18:33.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:18:34.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:18:34.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:18:34.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:18:34.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:18:34.265 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:18:34.266 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:18:34.266 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:18:34.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:18:34.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:18:34.267 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:18:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:18:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:18:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:18:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:18:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:18:34.371 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:18:34.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:18:34.590 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:18:34.590 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:18:34.590 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:18:34.590 INFO:tasks.workunit.client.0.trial166.stderr:4' 2026-02-21T11:18:34.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:18:34.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:18:34.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:18:34.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803782 2026-02-21T11:18:34.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803782 2026-02-21T11:18:34.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782' 2026-02-21T11:18:34.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:18:34.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:18:34.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574853 2026-02-21T11:18:34.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574853 2026-02-21T11:18:34.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574853' 2026-02-21T11:18:34.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:18:34.749 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:18:34.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411332 2026-02-21T11:18:34.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411332 2026-02-21T11:18:34.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-77309411332' 2026-02-21T11:18:34.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:18:34.825 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:18:34.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182405 2026-02-21T11:18:34.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182405 2026-02-21T11:18:34.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-77309411332 3-107374182405' 2026-02-21T11:18:34.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:18:34.908 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:18:34.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920771 2026-02-21T11:18:34.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920771 2026-02-21T11:18:34.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-77309411332 3-107374182405 4-141733920771' 2026-02-21T11:18:34.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:18:34.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803782 2026-02-21T11:18:34.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:18:34.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:18:34.988 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803782 2026-02-21T11:18:34.988 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:18:34.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803782 2026-02-21T11:18:34.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-21T11:18:34.990 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803782 2026-02-21T11:18:34.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:18:35.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-21T11:18:35.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:18:35.210 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574853 2026-02-21T11:18:35.210 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:18:35.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:18:35.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574853 2026-02-21T11:18:35.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:18:35.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574853 2026-02-21T11:18:35.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-21T11:18:35.213 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574853 2026-02-21T11:18:35.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:18:35.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-21T11:18:35.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:18:35.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411332 2026-02-21T11:18:35.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:18:35.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:18:35.441 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411332 2026-02-21T11:18:35.441 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:18:35.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411332 2026-02-21T11:18:35.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-21T11:18:35.443 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 77309411332 2026-02-21T11:18:35.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:18:35.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-21T11:18:35.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:18:35.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182405 2026-02-21T11:18:35.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:18:35.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:18:35.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182405 2026-02-21T11:18:35.665 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:18:35.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182405 2026-02-21T11:18:35.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-21T11:18:35.666 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182405 2026-02-21T11:18:35.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:18:35.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182403 -lt 107374182405 2026-02-21T11:18:35.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:18:36.884 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:18:36.884 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:18:37.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-21T11:18:37.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:18:37.104 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920771 2026-02-21T11:18:37.104 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:18:37.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:18:37.106 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920771 2026-02-21T11:18:37.106 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:18:37.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920771 2026-02-21T11:18:37.108 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920771 2026-02-21T11:18:37.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920771' 2026-02-21T11:18:37.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:18:37.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920771 -lt 141733920771 2026-02-21T11:18:37.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:18:37.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:18:37.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:18:37.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:18:37.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:18:37.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:18:37.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:18:37.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:18:37.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:18:37.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:18:37.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:18:37.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:18:37.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:18:37.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:18:37.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:18:38.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:18:38.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:18:38.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:18:38.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-21T11:18:38.318 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:33.723451+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-21T11:18:33.198093+0000 0'0 2026-02-21T11:18:33.198093+0000 0 0 periodic scrub scheduled @ 2026-02-22T21:01:55.754039+0000 0 0 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.711622+0000 0'0 41:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:51:15.439644+0000 0 0 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.711494+0000 0'0 41:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:35:53.262087+0000 0 0 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.594046+0000 0'0 41:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T21:41:00.370683+0000 0 0 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:24.614827+0000 0'0 41:65 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:56:08.455656+0000 0 0 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:18:38.319 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:18:38.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:18:38.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:18:38.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:38.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-21T11:18:38.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-21T11:18:38.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-21T11:18:38.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:18:38.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-21T11:18:38.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-21T11:18:38.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-21T11:18:38.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:18:38.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:18:38.568 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:18:38.568 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:18:38.786 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:18:38.786 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-21T11:18:38.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:18:38.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:18:38.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:18:38.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:18:38.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:18:38.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:18:39.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:18:39.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-21T11:18:39.304 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already down. 2026-02-21T11:18:39.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-21T11:18:39.560 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already out. 2026-02-21T11:18:39.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:33.723451+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-21T11:18:33.198093+0000 0'0 2026-02-21T11:18:33.198093+0000 0 0 periodic scrub scheduled @ 2026-02-22T21:01:55.754039+0000 0 0 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.711622+0000 0'0 41:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:51:15.439644+0000 0 0 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.711494+0000 0'0 41:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:35:53.262087+0000 0 0 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.594046+0000 0'0 41:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T21:41:00.370683+0000 0 0 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:24.614827+0000 0'0 41:65 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:56:08.455656+0000 0 0 2026-02-21T11:18:39.772 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:18:39.773 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:18:39.773 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:18:39.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-21T11:18:39.782 INFO:tasks.workunit.client.0.trial166.stderr:4+0 records in 2026-02-21T11:18:39.782 INFO:tasks.workunit.client.0.trial166.stderr:4+0 records out 2026-02-21T11:18:39.782 INFO:tasks.workunit.client.0.trial166.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000219058 s, 18.7 MB/s 2026-02-21T11:18:39.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-21T11:18:39.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.958 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.958 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:39.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:39.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.129 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.129 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.579 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.579 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.932 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.932 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:40.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:40.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.034 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:41.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:41.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.268 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.268 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.352 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.533 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.533 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.755 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.756 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:42.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:42.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.685 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.685 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.884 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:43.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:43.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.220 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.272 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:44.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:44.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:45.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:45.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.272 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.273 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.821 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:18:46.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-21T11:18:46.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-21T11:18:46.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:18:46.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:18:46.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:18:46.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj250 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj250 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:18:46.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-21T11:18:46.988 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-21T11:18:46.988 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:18:47.193 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:2147483647' 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:18:47.194 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:18:47.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:18:47.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:18:47.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:18:47.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:18:47.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:18:47.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:47.196 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:47.197 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:18:47.197 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:18:47.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:18:47.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:47.257 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:47.258 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:18:47.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:18:47.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj250 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-21T11:18:47.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:18:47.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj250 2026-02-21T11:18:47.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:18:47.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-21T11:18:47.317 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-21T11:18:47.317 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:2147483647' 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-21T11:18:47.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:18:47.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:18:47.539 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:18:47.539 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-21T11:18:47.539 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:18:47.539 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:18:47.540 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:47.540 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:47.540 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:47.540 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:18:47.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:18:47.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:18:47.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:18:47.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:18:47.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-21T11:18:47.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:18:47.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:18:47.601 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:18:47.601 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:47.601 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:47.601 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:18:47.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:18:47.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-21T11:18:47.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-21T11:18:47.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:18:47.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:18:47.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:18:47.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:18:47.661 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:18:47.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:18:47.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:18:47.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:18:47.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:18:47.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:18:47.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:18:47.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:18:47.683 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:47.675+0000 7fc67a380a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:47.684 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:47.679+0000 7fc67a380a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:47.685 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:47.679+0000 7fc67a380a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:18:47.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:18:47.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:18:47.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:18:47.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:18:47.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:18:47.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:47.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:18:47.875 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:18:47.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:47.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:18:48.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:18:48.133 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:48.127+0000 7fc67a380a40 -1 Falling back to public interface 2026-02-21T11:18:48.719 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:18:48.711+0000 7fc67a380a40 -1 osd.2 41 log_to_monitors true 2026-02-21T11:18:49.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:18:49.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:18:49.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:18:49.089 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:18:49.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:18:49.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:18:49.304 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up out weight 0 up_from 47 up_thru 19 down_at 42 last_clean_interval [18,41) [v2:127.0.0.1:6818/2901371131,v1:127.0.0.1:6819/2901371131] [v2:127.0.0.1:6820/2901371131,v1:127.0.0.1:6821/2901371131] exists,up 2a651e32-014f-4716-adb5-ba7a89280c5c 2026-02-21T11:18:49.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:18:49.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:18:49.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:18:49.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-21T11:18:49.539 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already in. 2026-02-21T11:18:49.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-21T11:19:04.552 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-21T11:19:04.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-21T11:19:04.555 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-21T11:19:04.555 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-21T11:19:04.555 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-21T11:19:04.556 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-21T11:19:04.556 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-21T11:19:04.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-21T11:19:04.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-21T11:19:04.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-21T11:19:04.761 INFO:tasks.workunit.client.0.trial166.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-21T11:19:04.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-21T11:19:04.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-21T11:19:04.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-21T11:18:50.093918+0000 45'301 54:909 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-21T11:18:33.198093+0000 0'0 2026-02-21T11:18:33.198093+0000 0 0 no scrub is scheduled 0 0 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:49.588719+0000 0'0 54:65 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:55:27.540731+0000 0 0 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.711494+0000 0'0 54:119 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:35:53.262087+0000 0 0 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.594046+0000 0'0 54:47 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T21:41:00.370683+0000 0 0 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:49.588714+0000 0'0 54:108 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T22:50:12.928004+0000 0 0 2026-02-21T11:19:04.970 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:19:04.971 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:19:04.971 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:19:04.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:19:04.981 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:19:04.981 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:19:04.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:19:04.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:19:04.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:19:05.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:19:05.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-21T11:19:10.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-21T11:19:10.492 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-21T11:19:05.138670+0000 45'301 56:921 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-21T11:18:33.198093+0000 0'0 2026-02-21T11:18:33.198093+0000 0 0 no scrub is scheduled 0 0 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-21T11:19:05.136487+0000 0'0 56:73 [4,1] 4 [4,1] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T18:24:38.611347+0000 0 0 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.711494+0000 0'0 56:123 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T16:35:53.262087+0000 0 0 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:18:31.594046+0000 0'0 56:51 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T21:41:00.370683+0000 0 0 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-21T11:19:05.136867+0000 0'0 56:116 [1,0] 1 [1,0] 1 0'0 2026-02-21T11:18:12.699481+0000 0'0 2026-02-21T11:18:12.699481+0000 0 0 periodic scrub scheduled @ 2026-02-22T19:55:10.623447+0000 0 0 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:19:10.493 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:19:10.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:19:10.591 INFO:tasks.workunit.client.0.trial166.stdout:{ 2026-02-21T11:19:10.591 INFO:tasks.workunit.client.0.trial166.stdout: "num_missing": 1, 2026-02-21T11:19:10.591 INFO:tasks.workunit.client.0.trial166.stdout: "num_unfound": 1, 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "objects": [ 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "oid": { 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "oid": "obj250", 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "key": "", 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "snapid": -2, 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "hash": 2249616407, 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "max": 0, 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "pool": 2, 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "namespace": "" 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "need": "45'251", 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "have": "0'0", 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "flags": "none", 2026-02-21T11:19:10.592 INFO:tasks.workunit.client.0.trial166.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "locations": [ 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: ] 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "state": "NotRecovering", 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "available_might_have_unfound": true, 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "might_have_unfound": [ 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "2(4)", 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: "status": "osd is down" 2026-02-21T11:19:10.593 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.594 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.594 INFO:tasks.workunit.client.0.trial166.stdout: "more": false 2026-02-21T11:19:10.594 INFO:tasks.workunit.client.0.trial166.stdout:} 2026-02-21T11:19:10.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout:{ 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: "snap_trimq": "[]", 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: "snap_trimq_len": 0, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: "epoch": 56, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.676 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "acting_recovery_backfill": [ 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "info": { 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s0", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "0", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'301", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'250", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "45'200", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 301, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "56", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 55, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 55, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 55, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'301", 2026-02-21T11:19:10.677 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 922, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 56, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:19:05.138670+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:19:05.138670+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:19:05.138670+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:19:05.138670+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:18:38.346621+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:19:05.136783+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:19:05.136783+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:19:05.138670+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:19:05.135247+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:19:05.135109+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 55, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "45'200", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "45'200", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 200, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:19:10.678 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 1232896, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 301, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 1505, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 1, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 1, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 303, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 1, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 301, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 301, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 1204, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:19:10.679 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 300 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),4(2)", 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 1 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:19:10.680 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "peer_info": [ 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "0(3)", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s3", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "3", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'301", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'301", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "45'200", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 301, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 55, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 55, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 55, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'301", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 912, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 55, 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+remapped", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:18:38.346621+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:18:38.956076+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:18:38.955975+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 55, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "45'200", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "45'200", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 200, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T11:42:03.921100+0000", 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 1232896, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 301, 2026-02-21T11:19:10.682 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 1505, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 301, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 301, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 301, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 1204, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.683 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 301 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "1(1)", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s1", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "1", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'301", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'301", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "45'200", 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 301, 2026-02-21T11:19:10.684 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 55, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 55, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 55, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'301", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 331, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 45, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+remapped", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:18:38.346621+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:18:38.956076+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:18:38.955975+0000", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 55, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "45'200", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "45'200", 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.685 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 200, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T11:42:03.921100+0000", 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 1232896, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 301, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 1505, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 1, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 301, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 301, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 301, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 1204, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:19:10.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.687 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 301 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "4(2)", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s2", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "2", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'301", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'301", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "45'200", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 301, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 55, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 55, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 55, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:19:10.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'301", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 331, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 45, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+remapped", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:18:38.346621+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:18:38.957590+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:18:46.980799+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:18:38.956076+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:18:38.955975+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 55, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "45'200", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "45'200", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:18:33.198093+0000", 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 200, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:19:10.689 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T11:42:03.921100+0000", 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 1232896, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 301, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 1505, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 301, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 301, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 301, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 1204, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:19:10.690 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: 2147483647 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 301 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 56, 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:19:10.691 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "recovery_state": [ 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "name": "Started/Primary/Active", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "enter_time": "2026-02-21T11:19:05.135125+0000", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "might_have_unfound": [ 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "0(3)", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "1(1)", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "2(4)", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "status": "osd is down" 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "4(2)", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "recovery_progress": { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "backfill_targets": [], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "waiting_on_backfill": [], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill_started": "MIN", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "backfill_info": { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "begin": "MIN", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "end": "MIN", 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "objects": [] 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "peer_backfill_info": [], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "backfills_in_flight": [], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "recovering": [], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "pg_backend": { 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "recovery_ops": [], 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: "read_ops": [] 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.692 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "name": "Started", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "enter_time": "2026-02-21T11:19:05.035439+0000" 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "scrubber": { 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "active": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "must_scrub": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "must_deep_scrub": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "must_repair": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "sched_time": "2026-02-22T13:01:12.056885+0000", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "schedule": "not registered for scrubbing", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "urgency": "periodic-regular", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "shallow-target": { 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s0", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "level": "shallow", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "urgency": "periodic-regular", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "sched_time": "2026-02-22T13:01:12.056885+0000", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "orig_sched_time": "2026-02-22T13:01:12.056885+0000", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "last_issue": "ok", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "forced": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "eligible": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "queued": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "active": false 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "deep-target": { 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s0", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "level": "deep", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "urgency": "periodic-regular", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "sched_time": "2026-02-27T12:45:06.112621+0000", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "orig_sched_time": "2026-02-27T12:45:06.112621+0000", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "last_issue": "ok", 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "forced": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "eligible": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "queued": false, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "active": false 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout: "agent_state": {} 2026-02-21T11:19:10.693 INFO:tasks.workunit.client.0.trial166.stdout:} 2026-02-21T11:19:10.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:19:10.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-21T11:19:10.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:19:10.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-21T11:19:10.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-21T11:19:10.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-21T11:19:10.858 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:19:10.858 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-21T11:19:10.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-21T11:19:10.935 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-21T11:19:10.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-21T11:19:10.936 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:19:10.936 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-21T11:19:11.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:19:11.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:11.019 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:19:11.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:19:11.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:19:11.021 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:19:11.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:19:11.036 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:11.031+0000 7f009c722a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:11.037 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:11.031+0000 7f009c722a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:11.038 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:11.031+0000 7f009c722a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:11.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:19:11.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:11.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:19:11.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:11.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:11.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:11.234 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:11.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:11.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:11.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:19:11.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:11.501 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:11.495+0000 7f009c722a40 -1 Falling back to public interface 2026-02-21T11:19:12.070 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:12.063+0000 7f009c722a40 -1 osd.2 54 log_to_monitors true 2026-02-21T11:19:12.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:12.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:12.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:12.451 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:12.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:12.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:19:12.663 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 58 up_thru 19 down_at 55 last_clean_interval [47,54) [v2:127.0.0.1:6818/1508727636,v1:127.0.0.1:6819/1508727636] [v2:127.0.0.1:6820/1508727636,v1:127.0.0.1:6821/1508727636] exists,up 2a651e32-014f-4716-adb5-ba7a89280c5c 2026-02-21T11:19:12.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:12.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:12.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:12.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-21T11:19:17.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-21T11:19:17.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-21T11:19:17.740 INFO:tasks.workunit.client.0.trial166.stderr:pg has no unfound objects 2026-02-21T11:19:17.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-21T11:19:17.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:19:17.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:19:17.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:19:17.749 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:19:17.751 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:19:17.751 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:19:17.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:19:17.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:19:17.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:19:17.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:19:17.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:19:17.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:19:17.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:19:17.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:19:17.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:19:18.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:19:18.068 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:19:18.069 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:19:18.069 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:19:18.069 INFO:tasks.workunit.client.0.trial166.stderr:4' 2026-02-21T11:19:18.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:19:18.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:18.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:19:18.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803793 2026-02-21T11:19:18.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803793 2026-02-21T11:19:18.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793' 2026-02-21T11:19:18.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:18.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:19:18.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574864 2026-02-21T11:19:18.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574864 2026-02-21T11:19:18.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-55834574864' 2026-02-21T11:19:18.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:18.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:19:18.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=249108103172 2026-02-21T11:19:18.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 249108103172 2026-02-21T11:19:18.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-249108103172' 2026-02-21T11:19:18.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:18.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:19:18.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182416 2026-02-21T11:19:18.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182416 2026-02-21T11:19:18.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-249108103172 3-107374182416' 2026-02-21T11:19:18.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:18.386 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:19:18.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920782 2026-02-21T11:19:18.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920782 2026-02-21T11:19:18.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-55834574864 2-249108103172 3-107374182416 4-141733920782' 2026-02-21T11:19:18.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:19:18.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803793 2026-02-21T11:19:18.466 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:19:18.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:19:18.466 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803793 2026-02-21T11:19:18.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:19:18.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803793 2026-02-21T11:19:18.468 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803793 2026-02-21T11:19:18.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-21T11:19:18.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:19:18.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803793 2026-02-21T11:19:18.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:19:19.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:19:19.678 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:19:19.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-21T11:19:19.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:19:19.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574864 2026-02-21T11:19:19.893 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:19:19.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:19:19.894 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574864 2026-02-21T11:19:19.894 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:19:19.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574864 2026-02-21T11:19:19.896 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574864 2026-02-21T11:19:19.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574864' 2026-02-21T11:19:19.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:19:20.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-21T11:19:20.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:19:20.114 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-249108103172 2026-02-21T11:19:20.114 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:19:20.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:19:20.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-249108103172 2026-02-21T11:19:20.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:19:20.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=249108103172 2026-02-21T11:19:20.117 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 249108103172 2026-02-21T11:19:20.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 249108103172' 2026-02-21T11:19:20.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:19:20.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 249108103172 -lt 249108103172 2026-02-21T11:19:20.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:19:20.332 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182416 2026-02-21T11:19:20.333 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:19:20.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:19:20.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182416 2026-02-21T11:19:20.335 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:19:20.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182416 2026-02-21T11:19:20.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182416' 2026-02-21T11:19:20.336 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182416 2026-02-21T11:19:20.337 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:19:20.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182416 -lt 107374182416 2026-02-21T11:19:20.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:19:20.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920782 2026-02-21T11:19:20.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:19:20.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:19:20.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920782 2026-02-21T11:19:20.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:19:20.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920782 2026-02-21T11:19:20.550 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920782 2026-02-21T11:19:20.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920782' 2026-02-21T11:19:20.551 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:19:20.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920782 -lt 141733920782 2026-02-21T11:19:20.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:19:20.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:19:20.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:19:21.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:19:21.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:19:21.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:19:21.267 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:19:21.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:19:21.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:19:21.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:19:21.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:19:21.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:19:21.545 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-21T11:19:21.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-21T11:19:21.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-21T11:19:21.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-21T11:19:21.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.606 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-21T11:19:21.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-21T11:19:21.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-21T11:19:21.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-21T11:19:21.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-21T11:19:21.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-21T11:19:21.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-21T11:19:21.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-21T11:19:21.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-21T11:19:21.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-21T11:19:21.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-21T11:19:21.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-21T11:19:21.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-21T11:19:21.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-21T11:19:21.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-21T11:19:21.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-21T11:19:21.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-21T11:19:21.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-21T11:19:21.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-21T11:19:21.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:21.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:21.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-21T11:19:21.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-21T11:19:22.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.036 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-21T11:19:22.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-21T11:19:22.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-21T11:19:22.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-21T11:19:22.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.129 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-21T11:19:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-21T11:19:22.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-21T11:19:22.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-21T11:19:22.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-21T11:19:22.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-21T11:19:22.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-21T11:19:22.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-21T11:19:22.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-21T11:19:22.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-21T11:19:22.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-21T11:19:22.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-21T11:19:22.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-21T11:19:22.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-21T11:19:22.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-21T11:19:22.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-21T11:19:22.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-21T11:19:22.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-21T11:19:22.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-21T11:19:22.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-21T11:19:22.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-21T11:19:22.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-21T11:19:22.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-21T11:19:22.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-21T11:19:22.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-21T11:19:22.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.673 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.673 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-21T11:19:22.673 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-21T11:19:22.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-21T11:19:22.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.741 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-21T11:19:22.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-21T11:19:22.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-21T11:19:22.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-21T11:19:22.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-21T11:19:22.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-21T11:19:22.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-21T11:19:22.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-21T11:19:22.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-21T11:19:22.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-21T11:19:22.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-21T11:19:22.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-21T11:19:22.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:22.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:22.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-21T11:19:22.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-21T11:19:23.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-21T11:19:23.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-21T11:19:23.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-21T11:19:23.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.102 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-21T11:19:23.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-21T11:19:23.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-21T11:19:23.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-21T11:19:23.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-21T11:19:23.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-21T11:19:23.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-21T11:19:23.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-21T11:19:23.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-21T11:19:23.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-21T11:19:23.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-21T11:19:23.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-21T11:19:23.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-21T11:19:23.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-21T11:19:23.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-21T11:19:23.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-21T11:19:23.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-21T11:19:23.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-21T11:19:23.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-21T11:19:23.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-21T11:19:23.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-21T11:19:23.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.563 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-21T11:19:23.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-21T11:19:23.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-21T11:19:23.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-21T11:19:23.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-21T11:19:23.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-21T11:19:23.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-21T11:19:23.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-21T11:19:23.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-21T11:19:23.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-21T11:19:23.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-21T11:19:23.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-21T11:19:23.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-21T11:19:23.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-21T11:19:23.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-21T11:19:23.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-21T11:19:23.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-21T11:19:23.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.941 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-21T11:19:23.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-21T11:19:23.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-21T11:19:23.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:23.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:23.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-21T11:19:23.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-21T11:19:24.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-21T11:19:24.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-21T11:19:24.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-21T11:19:24.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-21T11:19:24.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-21T11:19:24.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-21T11:19:24.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-21T11:19:24.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-21T11:19:24.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-21T11:19:24.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-21T11:19:24.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-21T11:19:24.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-21T11:19:24.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-21T11:19:24.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-21T11:19:24.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-21T11:19:24.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-21T11:19:24.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-21T11:19:24.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-21T11:19:24.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-21T11:19:24.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-21T11:19:24.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-21T11:19:24.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-21T11:19:24.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-21T11:19:24.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-21T11:19:24.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-21T11:19:24.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-21T11:19:24.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-21T11:19:24.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-21T11:19:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-21T11:19:24.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-21T11:19:24.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-21T11:19:24.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-21T11:19:24.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-21T11:19:24.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-21T11:19:24.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-21T11:19:24.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-21T11:19:24.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-21T11:19:24.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-21T11:19:24.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-21T11:19:24.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-21T11:19:24.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-21T11:19:24.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-21T11:19:24.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-21T11:19:24.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-21T11:19:24.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:24.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:24.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-21T11:19:24.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-21T11:19:25.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-21T11:19:25.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-21T11:19:25.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-21T11:19:25.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-21T11:19:25.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-21T11:19:25.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-21T11:19:25.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-21T11:19:25.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-21T11:19:25.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-21T11:19:25.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-21T11:19:25.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-21T11:19:25.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-21T11:19:25.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-21T11:19:25.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-21T11:19:25.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-21T11:19:25.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-21T11:19:25.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-21T11:19:25.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-21T11:19:25.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-21T11:19:25.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-21T11:19:25.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-21T11:19:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-21T11:19:25.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-21T11:19:25.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-21T11:19:25.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-21T11:19:25.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-21T11:19:25.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-21T11:19:25.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-21T11:19:25.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-21T11:19:25.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-21T11:19:25.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-21T11:19:25.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-21T11:19:25.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-21T11:19:25.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-21T11:19:25.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-21T11:19:25.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-21T11:19:25.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.893 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-21T11:19:25.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-21T11:19:25.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.941 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-21T11:19:25.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-21T11:19:25.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-21T11:19:25.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:25.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:25.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-21T11:19:25.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-21T11:19:26.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-21T11:19:26.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-21T11:19:26.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-21T11:19:26.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-21T11:19:26.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-21T11:19:26.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-21T11:19:26.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-21T11:19:26.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-21T11:19:26.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-21T11:19:26.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.240 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.241 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.241 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-21T11:19:26.241 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-21T11:19:26.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-21T11:19:26.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-21T11:19:26.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-21T11:19:26.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-21T11:19:26.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-21T11:19:26.380 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-21T11:19:26.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-21T11:19:26.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-21T11:19:26.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-21T11:19:26.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-21T11:19:26.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-21T11:19:26.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.530 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-21T11:19:26.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-21T11:19:26.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-21T11:19:26.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-21T11:19:26.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-21T11:19:26.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-21T11:19:26.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-21T11:19:26.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-21T11:19:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-21T11:19:26.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-21T11:19:26.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.743 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-21T11:19:26.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-21T11:19:26.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-21T11:19:26.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.815 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.815 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-21T11:19:26.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-21T11:19:26.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-21T11:19:26.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-21T11:19:26.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-21T11:19:26.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-21T11:19:26.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-21T11:19:26.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-21T11:19:26.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-21T11:19:26.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:26.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:26.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-21T11:19:26.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-21T11:19:27.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-21T11:19:27.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-21T11:19:27.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-21T11:19:27.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-21T11:19:27.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-21T11:19:27.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.155 INFO:tasks.workunit.client.0.trial166.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-21T11:19:27.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-21T11:19:27.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-21T11:19:27.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-21T11:19:27.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-21T11:19:27.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-21T11:19:27.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.273 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.273 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-21T11:19:27.273 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-21T11:19:27.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-21T11:19:27.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-21T11:19:27.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-21T11:19:27.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-21T11:19:27.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-21T11:19:27.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-21T11:19:27.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-21T11:19:27.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.487 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.487 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-21T11:19:27.487 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-21T11:19:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.533 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.533 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-21T11:19:27.533 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-21T11:19:27.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-21T11:19:27.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-21T11:19:27.607 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-21T11:19:27.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-21T11:19:27.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-21T11:19:27.678 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-21T11:19:27.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-21T11:19:27.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-21T11:19:27.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-21T11:19:27.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-21T11:19:27.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-21T11:19:27.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-21T11:19:27.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-21T11:19:27.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-21T11:19:27.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-21T11:19:27.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-21T11:19:27.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-21T11:19:27.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-21T11:19:27.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-21T11:19:27.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:27.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:27.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-21T11:19:27.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-21T11:19:28.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-21T11:19:28.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-21T11:19:28.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-21T11:19:28.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-21T11:19:28.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-21T11:19:28.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-21T11:19:28.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-21T11:19:28.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-21T11:19:28.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-21T11:19:28.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-21T11:19:28.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-21T11:19:28.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:19:28.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-21T11:19:28.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-21T11:19:28.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:19:28.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:19:28.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:19:28.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:19:28.559 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:19:28.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:19:28.839 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:19:28.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:19:28.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:19:28.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:19:28.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:19:28.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:19:28.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:19:28.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:19:28.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:19:28.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:19:28.963 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:19:28.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:19:28.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:19:28.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:19:28.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:19:28.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:19:28.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:19:28.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:19:29.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:19:29.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:19:29.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:19:29.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:19:29.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:19:29.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:19:29.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:19:29.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:19:29.016 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:19:29.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:19:29.017 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:19:29.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:19:29.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:19:29.019 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:19:29.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:19:29.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:19:29.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:19:29.021 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:19:29.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:19:29.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:19:29.023 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:19:29.024 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:19:29.024 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.024 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:19:29.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:19:29.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:19:29.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:19:29.027 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:19:29.027 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.027 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:19:29.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:19:29.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:19:29.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:19:29.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:19:29.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:19:29.051 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:19:29.051 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:29.051 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:29.051 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:29.052 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.052 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.052 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:29.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:19:29.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:19:29.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:19:29.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:19:29.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:19:29.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.082 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.083 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:19:29.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:19:29.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:19:29.137 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:19:29.138 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:19:29.138 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.138 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.138 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:19:29.138 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:19:29.138 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:19:29.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:19:29.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:19:29.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:29.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:29.303 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:29.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:29.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:29.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:29.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:19:29.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:19:29.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:19:29.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:19:29.433 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:19:29.441 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:19:30.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:19:30.492 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:19:30.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:19:30.499 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:19:29.063+0000 7fdeed386ec0 0 load: jerasure load: lrc 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:19:30.500 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-21T11:19:30.501 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-21T11:19:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:19:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:30.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:30.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:30.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:30.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:30.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:30.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:19:30.507 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:30.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bc909095-7972-4e5b-a675-83a519ef19c1 2026-02-21T11:19:30.508 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 bc909095-7972-4e5b-a675-83a519ef19c1 2026-02-21T11:19:30.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bc909095-7972-4e5b-a675-83a519ef19c1' 2026-02-21T11:19:30.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:30.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDClJlp88mvHhAAAyGcGY0ys1lju4WG5sIzRQ== 2026-02-21T11:19:30.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDClJlp88mvHhAAAyGcGY0ys1lju4WG5sIzRQ=="}' 2026-02-21T11:19:30.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bc909095-7972-4e5b-a675-83a519ef19c1 -i td/test-erasure-eio/0/new.json 2026-02-21T11:19:30.628 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:30.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:19:30.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDClJlp88mvHhAAAyGcGY0ys1lju4WG5sIzRQ== --osd-uuid bc909095-7972-4e5b-a675-83a519ef19c1 2026-02-21T11:19:30.653 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:30.647+0000 7f8df1796a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:30.654 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:30.647+0000 7f8df1796a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:30.655 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:30.647+0000 7f8df1796a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:30.655 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:30.647+0000 7f8df1796a40 -1 bdev(0x55a0cd712400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:30.655 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:30.647+0000 7f8df1796a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:19:31.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:19:31.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:31.304 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:19:31.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:19:31.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:31.415 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:19:31.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:19:31.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:31.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:31.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:31.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:31.430 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:31.423+0000 7efc3d538a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:31.431 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:31.427+0000 7efc3d538a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:31.432 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:31.427+0000 7efc3d538a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:31.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:19:31.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:31.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:19:31.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:31.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:31.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:31.576 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:31.576 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:31.576 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:31.576 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:19:31.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:31.861 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:31.855+0000 7efc3d538a40 -1 Falling back to public interface 2026-02-21T11:19:32.413 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:32.407+0000 7efc3d538a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:19:32.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:32.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:32.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:32.689 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:32.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:32.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:19:32.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:33.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:33.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:33.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:19:33.903 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:19:33.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:33.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:19:34.116 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/653956288,v1:127.0.0.1:6803/653956288] [v2:127.0.0.1:6804/653956288,v1:127.0.0.1:6805/653956288] exists,up bc909095-7972-4e5b-a675-83a519ef19c1 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:19:34.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:34.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:19:34.119 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:34.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=38fad490-cdb4-48a2-b2c2-679e3d97df25 2026-02-21T11:19:34.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 38fad490-cdb4-48a2-b2c2-679e3d97df25' 2026-02-21T11:19:34.120 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 38fad490-cdb4-48a2-b2c2-679e3d97df25 2026-02-21T11:19:34.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:34.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDGlJlpxNLGBxAAQci7dDBMAt1WugYlrLIKUA== 2026-02-21T11:19:34.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDGlJlpxNLGBxAAQci7dDBMAt1WugYlrLIKUA=="}' 2026-02-21T11:19:34.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 38fad490-cdb4-48a2-b2c2-679e3d97df25 -i td/test-erasure-eio/1/new.json 2026-02-21T11:19:34.348 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:34.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:19:34.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDGlJlpxNLGBxAAQci7dDBMAt1WugYlrLIKUA== --osd-uuid 38fad490-cdb4-48a2-b2c2-679e3d97df25 2026-02-21T11:19:34.384 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:34.379+0000 7fbf2682ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:34.385 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:34.379+0000 7fbf2682ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:34.386 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:34.379+0000 7fbf2682ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:34.387 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:34.379+0000 7fbf2682ba40 -1 bdev(0x55581cd4e400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:34.387 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:34.379+0000 7fbf2682ba40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:19:35.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:19:35.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:35.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:19:35.044 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:19:35.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:35.341 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:19:35.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:19:35.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:35.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:35.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:35.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:35.365 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:35.359+0000 7f8362636a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:35.367 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:35.359+0000 7f8362636a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:35.368 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:35.363+0000 7f8362636a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:35.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:19:35.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:35.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:19:35.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:35.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:35.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:35.553 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:35.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:35.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:35.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:19:35.725 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:35.719+0000 7f8362636a40 -1 Falling back to public interface 2026-02-21T11:19:35.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:36.329 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:36.323+0000 7f8362636a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:19:36.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:36.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:36.768 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:36.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:36.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:36.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:19:36.980 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3235572589,v1:127.0.0.1:6811/3235572589] [v2:127.0.0.1:6812/3235572589,v1:127.0.0.1:6813/3235572589] exists,up 38fad490-cdb4-48a2-b2c2-679e3d97df25 2026-02-21T11:19:36.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:36.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:36.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:36.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:36.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:36.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:19:36.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:36.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=111cb0d0-5a0f-41ce-873c-9baf9bedec5e 2026-02-21T11:19:36.987 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 111cb0d0-5a0f-41ce-873c-9baf9bedec5e 2026-02-21T11:19:36.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 111cb0d0-5a0f-41ce-873c-9baf9bedec5e' 2026-02-21T11:19:36.988 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:36.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIlJlpAahTOxAA9OwiAhNvKkbX1/XniW+WOA== 2026-02-21T11:19:36.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIlJlpAahTOxAA9OwiAhNvKkbX1/XniW+WOA=="}' 2026-02-21T11:19:36.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 111cb0d0-5a0f-41ce-873c-9baf9bedec5e -i td/test-erasure-eio/2/new.json 2026-02-21T11:19:37.220 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:19:37.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:19:37.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDIlJlpAahTOxAA9OwiAhNvKkbX1/XniW+WOA== --osd-uuid 111cb0d0-5a0f-41ce-873c-9baf9bedec5e 2026-02-21T11:19:37.248 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:37.243+0000 7ff9f39c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:37.249 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:37.243+0000 7ff9f39c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:37.251 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:37.243+0000 7ff9f39c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:37.252 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:37.243+0000 7ff9f39c1a40 -1 bdev(0x55e04feea400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:37.252 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:37.247+0000 7ff9f39c1a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:19:37.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:19:37.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:37.887 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:19:37.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:19:37.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:38.181 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:19:38.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:19:38.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:38.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:38.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:38.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:38.205 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:38.199+0000 7f871918ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:38.207 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:38.199+0000 7f871918ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:38.208 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:38.203+0000 7f871918ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:38.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:19:38.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:38.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:19:38.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:38.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:38.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:38.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:38.394 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:38.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:38.395 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:19:38.581 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:38.575+0000 7f871918ca40 -1 Falling back to public interface 2026-02-21T11:19:38.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:39.221 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:39.215+0000 7f871918ca40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:19:39.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:39.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:39.612 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:39.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:39.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:39.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:19:39.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:40.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:40.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:40.832 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:19:40.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:19:40.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:40.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:19:41.046 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1190652194,v1:127.0.0.1:6819/1190652194] [v2:127.0.0.1:6820/1190652194,v1:127.0.0.1:6821/1190652194] exists,up 111cb0d0-5a0f-41ce-873c-9baf9bedec5e 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:41.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:41.048 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:41.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:41.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:41.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:41.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:41.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:41.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:19:41.052 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:41.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2d55e1ab-9cdf-4381-880b-e5ae9937d492 2026-02-21T11:19:41.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2d55e1ab-9cdf-4381-880b-e5ae9937d492' 2026-02-21T11:19:41.052 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 2d55e1ab-9cdf-4381-880b-e5ae9937d492 2026-02-21T11:19:41.052 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:41.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNlJlpjHuVAxAAsXkYsEO/2wlzS0WjNxujgA== 2026-02-21T11:19:41.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNlJlpjHuVAxAAsXkYsEO/2wlzS0WjNxujgA=="}' 2026-02-21T11:19:41.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2d55e1ab-9cdf-4381-880b-e5ae9937d492 -i td/test-erasure-eio/3/new.json 2026-02-21T11:19:41.279 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:19:41.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:19:41.290 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNlJlpjHuVAxAAsXkYsEO/2wlzS0WjNxujgA== --osd-uuid 2d55e1ab-9cdf-4381-880b-e5ae9937d492 2026-02-21T11:19:41.306 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:41.299+0000 7f3b846eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:41.308 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:41.303+0000 7f3b846eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:41.309 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:41.303+0000 7f3b846eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:41.309 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:41.303+0000 7f3b846eba40 -1 bdev(0x5574fbdbc400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:41.310 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:41.303+0000 7f3b846eba40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:19:41.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:19:41.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:41.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:19:41.972 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:19:41.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:42.266 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:19:42.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:19:42.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:42.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:42.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:42.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:42.291 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:42.283+0000 7fe612348a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:42.292 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:42.287+0000 7fe612348a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:42.293 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:42.287+0000 7fe612348a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:42.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:19:42.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:42.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:19:42.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:42.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:42.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:42.482 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:42.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:42.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:42.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:19:42.661 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:42.655+0000 7fe612348a40 -1 Falling back to public interface 2026-02-21T11:19:42.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:43.305 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:43.299+0000 7fe612348a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:19:43.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:43.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:43.689 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:43.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:43.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:43.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:19:43.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:44.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:44.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:44.900 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:19:44.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:19:44.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:44.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:19:45.118 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 24 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2853861366,v1:127.0.0.1:6827/2853861366] [v2:127.0.0.1:6828/2853861366,v1:127.0.0.1:6829/2853861366] exists,up 2d55e1ab-9cdf-4381-880b-e5ae9937d492 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:45.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:45.120 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:19:45.121 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:45.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=729bfdd7-bd38-4cc9-bb97-13816b7427cb 2026-02-21T11:19:45.122 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 729bfdd7-bd38-4cc9-bb97-13816b7427cb 2026-02-21T11:19:45.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 729bfdd7-bd38-4cc9-bb97-13816b7427cb' 2026-02-21T11:19:45.123 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:45.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDRlJlpfF3fBxAATaAPcpzs+z/W2yM5MmmWiQ== 2026-02-21T11:19:45.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDRlJlpfF3fBxAATaAPcpzs+z/W2yM5MmmWiQ=="}' 2026-02-21T11:19:45.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 729bfdd7-bd38-4cc9-bb97-13816b7427cb -i td/test-erasure-eio/4/new.json 2026-02-21T11:19:45.361 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:19:45.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:19:45.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRlJlpfF3fBxAATaAPcpzs+z/W2yM5MmmWiQ== --osd-uuid 729bfdd7-bd38-4cc9-bb97-13816b7427cb 2026-02-21T11:19:45.388 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:45.383+0000 7f6eb0cd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:45.390 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:45.383+0000 7f6eb0cd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:45.391 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:45.383+0000 7f6eb0cd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:45.392 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:45.387+0000 7f6eb0cd1a40 -1 bdev(0x55a45dd50400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:45.392 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:45.387+0000 7f6eb0cd1a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:19:46.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:19:46.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:46.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:19:46.007 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:19:46.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:46.295 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:19:46.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:19:46.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:46.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:46.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:46.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:46.312 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:46.306+0000 7fd2388f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:46.314 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:46.306+0000 7fd2388f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:46.315 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:46.306+0000 7fd2388f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:46.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:19:46.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:46.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:19:46.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:46.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:46.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:46.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:46.506 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:46.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:46.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:19:46.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:46.729 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:46.722+0000 7fd2388f9a40 -1 Falling back to public interface 2026-02-21T11:19:47.317 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:47.310+0000 7fd2388f9a40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:19:47.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:47.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:47.730 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:47.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:47.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:47.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:19:47.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:48.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:48.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:48.942 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:19:48.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:19:48.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:48.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:19:49.149 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/602876985,v1:127.0.0.1:6835/602876985] [v2:127.0.0.1:6836/602876985,v1:127.0.0.1:6837/602876985] exists,up 729bfdd7-bd38-4cc9-bb97-13816b7427cb 2026-02-21T11:19:49.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:49.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:49.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:49.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a882bb05-b936-401b-b8d2-40de95028384 2026-02-21T11:19:49.158 INFO:tasks.workunit.client.0.trial166.stdout:add osd5 a882bb05-b936-401b-b8d2-40de95028384 2026-02-21T11:19:49.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 a882bb05-b936-401b-b8d2-40de95028384' 2026-02-21T11:19:49.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:49.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDVlJlpcivACRAAonDdu2cp+wRwnWIwYi/OKA== 2026-02-21T11:19:49.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDVlJlpcivACRAAonDdu2cp+wRwnWIwYi/OKA=="}' 2026-02-21T11:19:49.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a882bb05-b936-401b-b8d2-40de95028384 -i td/test-erasure-eio/5/new.json 2026-02-21T11:19:49.390 INFO:tasks.workunit.client.0.trial166.stdout:5 2026-02-21T11:19:49.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-21T11:19:49.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDVlJlpcivACRAAonDdu2cp+wRwnWIwYi/OKA== --osd-uuid a882bb05-b936-401b-b8d2-40de95028384 2026-02-21T11:19:49.425 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:49.418+0000 7fb7acae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:49.427 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:49.418+0000 7fb7acae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:49.428 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:49.422+0000 7fb7acae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:49.429 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:49.422+0000 7fb7acae7a40 -1 bdev(0x55ab81700400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:49.429 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:49.422+0000 7fb7acae7a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-21T11:19:50.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-21T11:19:50.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:50.076 INFO:tasks.workunit.client.0.trial166.stdout:adding osd5 key to auth repository 2026-02-21T11:19:50.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-21T11:19:50.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:50.363 INFO:tasks.workunit.client.0.trial166.stdout:start osd.5 2026-02-21T11:19:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-21T11:19:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:50.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:50.378 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:50.370+0000 7f230e4bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:50.379 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:50.374+0000 7f230e4bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:50.381 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:50.374+0000 7f230e4bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:50.578 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-21T11:19:50.578 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:50.578 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:19:50.579 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:50.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:50.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:50.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:50.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:50.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:50.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:19:50.789 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:50.782+0000 7f230e4bca40 -1 Falling back to public interface 2026-02-21T11:19:50.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:51.364 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:51.358+0000 7f230e4bca40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:19:51.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:51.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:51.794 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:51.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:51.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:51.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:19:52.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:53.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:53.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:53.010 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:19:53.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:19:53.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:53.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:19:53.220 INFO:tasks.workunit.client.0.trial166.stdout:osd.5 up in weight 1 up_from 41 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1906868904,v1:127.0.0.1:6843/1906868904] [v2:127.0.0.1:6844/1906868904,v1:127.0.0.1:6845/1906868904] exists,up a882bb05-b936-401b-b8d2-40de95028384 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-21T11:19:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:19:53.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-21T11:19:53.223 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:19:53.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e627f4dd-3c57-4347-bc36-117bd91a27f6 2026-02-21T11:19:53.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 e627f4dd-3c57-4347-bc36-117bd91a27f6' 2026-02-21T11:19:53.224 INFO:tasks.workunit.client.0.trial166.stdout:add osd6 e627f4dd-3c57-4347-bc36-117bd91a27f6 2026-02-21T11:19:53.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:19:53.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDZlJlpVn4FDhAA/gGCFvMuTX46Qs7BxeuMYA== 2026-02-21T11:19:53.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDZlJlpVn4FDhAA/gGCFvMuTX46Qs7BxeuMYA=="}' 2026-02-21T11:19:53.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e627f4dd-3c57-4347-bc36-117bd91a27f6 -i td/test-erasure-eio/6/new.json 2026-02-21T11:19:53.457 INFO:tasks.workunit.client.0.trial166.stdout:6 2026-02-21T11:19:53.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-21T11:19:53.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDZlJlpVn4FDhAA/gGCFvMuTX46Qs7BxeuMYA== --osd-uuid e627f4dd-3c57-4347-bc36-117bd91a27f6 2026-02-21T11:19:53.485 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:53.478+0000 7fab5ab84a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:53.486 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:53.478+0000 7fab5ab84a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:53.487 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:53.478+0000 7fab5ab84a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:53.488 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:53.482+0000 7fab5ab84a40 -1 bdev(0x55c95b21c400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-21T11:19:53.488 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:53.482+0000 7fab5ab84a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-21T11:19:54.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-21T11:19:54.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:19:54.159 INFO:tasks.workunit.client.0.trial166.stdout:adding osd6 key to auth repository 2026-02-21T11:19:54.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-21T11:19:54.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:19:54.454 INFO:tasks.workunit.client.0.trial166.stdout:start osd.6 2026-02-21T11:19:54.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-21T11:19:54.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:19:54.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:19:54.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:19:54.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:19:54.478 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:54.470+0000 7f0959e24a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:54.481 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:54.474+0000 7f0959e24a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:54.482 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:54.474+0000 7f0959e24a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:19:54.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-21T11:19:54.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:19:54.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:19:54.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:19:54.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:19:54.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:54.660 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:19:54.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:19:54.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:54.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:19:54.809 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:54.802+0000 7f0959e24a40 -1 Falling back to public interface 2026-02-21T11:19:54.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:19:55.432 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:19:55.426+0000 7f0959e24a40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:19:55.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:19:55.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:19:55.880 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:19:55.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:19:55.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:19:55.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:19:56.103 INFO:tasks.workunit.client.0.trial166.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3437216302,v1:127.0.0.1:6851/3437216302] [v2:127.0.0.1:6852/3437216302,v1:127.0.0.1:6853/3437216302] exists,up e627f4dd-3c57-4347-bc36-117bd91a27f6 2026-02-21T11:19:56.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:19:56.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:19:56.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:19:56.168 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:19:56.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:19:56.176 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:19:31.859+0000 7efc3d538a40 0 load: jerasure load: lrc 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:19:56.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-21T11:19:56.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:19:56.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:19:56.850 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:19:56.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:19:57.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:19:57.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:19:57.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:19:57.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:19:57.863 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:19:57.864 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:19:57.864 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:19:57.864 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:19:57.864 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:19:57.864 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:19:57.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:19:57.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:19:57.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:19:57.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:19:57.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:19:57.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:19:58.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:19:58.172 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:19:58.172 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.173 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:19:58.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803783 2026-02-21T11:19:58.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803783 2026-02-21T11:19:58.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783' 2026-02-21T11:19:58.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.250 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:19:58.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607559 2026-02-21T11:19:58.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607559 2026-02-21T11:19:58.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559' 2026-02-21T11:19:58.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.330 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:19:58.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444038 2026-02-21T11:19:58.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444038 2026-02-21T11:19:58.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038' 2026-02-21T11:19:58.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.410 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:19:58.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=103079215110 2026-02-21T11:19:58.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 103079215110 2026-02-21T11:19:58.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-103079215110' 2026-02-21T11:19:58.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:19:58.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920773 2026-02-21T11:19:58.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920773 2026-02-21T11:19:58.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-103079215110 4-141733920773' 2026-02-21T11:19:58.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:19:58.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659140 2026-02-21T11:19:58.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659140 2026-02-21T11:19:58.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-103079215110 4-141733920773 5-176093659140' 2026-02-21T11:19:58.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:19:58.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:19:58.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364803 2026-02-21T11:19:58.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364803 2026-02-21T11:19:58.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-103079215110 4-141733920773 5-176093659140 6-214748364803' 2026-02-21T11:19:58.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:19:58.728 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803783 2026-02-21T11:19:58.728 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:19:58.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:19:58.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803783 2026-02-21T11:19:58.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:19:58.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803783 2026-02-21T11:19:58.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-21T11:19:58.732 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803783 2026-02-21T11:19:58.732 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:19:58.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-21T11:19:58.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:19:59.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:19:59.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:20:00.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803783 2026-02-21T11:20:00.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:00.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607559 2026-02-21T11:20:00.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:00.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:20:00.166 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607559 2026-02-21T11:20:00.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:00.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607559 2026-02-21T11:20:00.168 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607559 2026-02-21T11:20:00.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-21T11:20:00.169 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:20:00.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-21T11:20:00.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:00.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444038 2026-02-21T11:20:00.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:00.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:20:00.377 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444038 2026-02-21T11:20:00.377 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:00.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444038 2026-02-21T11:20:00.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444038' 2026-02-21T11:20:00.378 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444038 2026-02-21T11:20:00.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:20:00.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444038 -lt 73014444038 2026-02-21T11:20:00.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:00.588 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-103079215110 2026-02-21T11:20:00.588 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:00.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:20:00.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-103079215110 2026-02-21T11:20:00.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:00.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=103079215110 2026-02-21T11:20:00.591 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 103079215110 2026-02-21T11:20:00.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 103079215110' 2026-02-21T11:20:00.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:20:00.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 103079215111 -lt 103079215110 2026-02-21T11:20:00.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:00.804 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920773 2026-02-21T11:20:00.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:00.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:20:00.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920773 2026-02-21T11:20:00.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:00.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920773 2026-02-21T11:20:00.808 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920773 2026-02-21T11:20:00.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920773' 2026-02-21T11:20:00.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:20:01.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920773 -lt 141733920773 2026-02-21T11:20:01.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:01.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-176093659140 2026-02-21T11:20:01.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:01.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:20:01.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-176093659140 2026-02-21T11:20:01.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:01.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659140 2026-02-21T11:20:01.031 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 176093659140 2026-02-21T11:20:01.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 176093659140' 2026-02-21T11:20:01.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:20:01.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659140 -lt 176093659140 2026-02-21T11:20:01.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:01.240 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-214748364803 2026-02-21T11:20:01.240 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:01.241 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:20:01.242 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-214748364803 2026-02-21T11:20:01.242 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:01.243 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364803 2026-02-21T11:20:01.243 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-21T11:20:01.244 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 214748364803 2026-02-21T11:20:01.244 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:20:01.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-21T11:20:01.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:20:01.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:01.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:20:01.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:20:01.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:20:01.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:20:01.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:20:01.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:20:01.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:20:01.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:20:01.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:20:01.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:20:01.959 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:20:01.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:01.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:20:02.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-21T11:20:02.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-21T11:20:02.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-21T11:20:02.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:20:02.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:20:02.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:20:02.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:20:02.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:20:02.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:02.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:20:02.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:02.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:20:02.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:02.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:20:02.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-21T11:20:03.129 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=mdata 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:20:03.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:20:03.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=5 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:20:03.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/5/type 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:20:03.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:03.352 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.5.asok 2026-02-21T11:20:03.353 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:20:03.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:20:03.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:20:03.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:20:03.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.5 2026-02-21T11:20:03.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-21T11:20:03.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-21T11:20:03.417 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:03.418 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:03.418 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:03.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.5.asok 2026-02-21T11:20:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:20:03.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-21T11:20:03.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:20:03.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:03.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:20:03.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:20:03.483 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:20:03.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:20:03.500 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:03.494+0000 7f1dcd7cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:03.501 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:03.494+0000 7f1dcd7cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:03.502 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:03.494+0000 7f1dcd7cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:03.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T11:20:03.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:03.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:20:03.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:03.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:03.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:03.695 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:03.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:03.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:03.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:20:03.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:03.925 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:03.918+0000 7f1dcd7cea40 -1 Falling back to public interface 2026-02-21T11:20:04.611 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:04.602+0000 7f1dcd7cea40 -1 osd.3 57 log_to_monitors true 2026-02-21T11:20:04.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:04.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:04.918 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:04.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:04.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:04.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:20:05.126 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 60 up_thru 60 down_at 57 last_clean_interval [24,57) [v2:127.0.0.1:6826/2798367394,v1:127.0.0.1:6827/2798367394] [v2:127.0.0.1:6828/2798367394,v1:127.0.0.1:6829/2798367394] exists,up 2d55e1ab-9cdf-4381-880b-e5ae9937d492 2026-02-21T11:20:05.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:05.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:05.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:20:05.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:20:05.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:20:05.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:20:05.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:20:05.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:20:05.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:20:05.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:20:05.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:20:05.447 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.448 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:20:05.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803787 2026-02-21T11:20:05.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803787 2026-02-21T11:20:05.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787' 2026-02-21T11:20:05.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.527 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:20:05.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607562 2026-02-21T11:20:05.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607562 2026-02-21T11:20:05.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562' 2026-02-21T11:20:05.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.608 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:20:05.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444042 2026-02-21T11:20:05.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444042 2026-02-21T11:20:05.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444042' 2026-02-21T11:20:05.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.686 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:20:05.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=257698037763 2026-02-21T11:20:05.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 257698037763 2026-02-21T11:20:05.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444042 3-257698037763' 2026-02-21T11:20:05.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.766 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:20:05.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920776 2026-02-21T11:20:05.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920776 2026-02-21T11:20:05.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444042 3-257698037763 4-141733920776' 2026-02-21T11:20:05.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.848 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:20:05.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659143 2026-02-21T11:20:05.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659143 2026-02-21T11:20:05.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444042 3-257698037763 4-141733920776 5-176093659143' 2026-02-21T11:20:05.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:05.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:20:06.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364806 2026-02-21T11:20:06.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364806 2026-02-21T11:20:06.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444042 3-257698037763 4-141733920776 5-176093659143 6-214748364806' 2026-02-21T11:20:06.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:06.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803787 2026-02-21T11:20:06.011 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:06.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:20:06.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803787 2026-02-21T11:20:06.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:06.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803787 2026-02-21T11:20:06.014 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803787 2026-02-21T11:20:06.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-21T11:20:06.015 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:20:06.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-21T11:20:06.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:06.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607562 2026-02-21T11:20:06.231 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:06.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:20:06.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607562 2026-02-21T11:20:06.233 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:06.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607562 2026-02-21T11:20:06.234 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607562 2026-02-21T11:20:06.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-21T11:20:06.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:20:06.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607560 -lt 51539607562 2026-02-21T11:20:06.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:20:07.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:20:07.462 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:20:07.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607563 -lt 51539607562 2026-02-21T11:20:07.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:07.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444042 2026-02-21T11:20:07.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:07.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:20:07.673 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444042 2026-02-21T11:20:07.673 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:07.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444042 2026-02-21T11:20:07.674 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444042 2026-02-21T11:20:07.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444042' 2026-02-21T11:20:07.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:20:07.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-21T11:20:07.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:07.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-257698037763 2026-02-21T11:20:07.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:07.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:20:07.901 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-257698037763 2026-02-21T11:20:07.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:07.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=257698037763 2026-02-21T11:20:07.903 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 257698037763 2026-02-21T11:20:07.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 257698037763' 2026-02-21T11:20:07.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:20:08.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 257698037763 -lt 257698037763 2026-02-21T11:20:08.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:08.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920776 2026-02-21T11:20:08.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:08.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:20:08.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920776 2026-02-21T11:20:08.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:08.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920776 2026-02-21T11:20:08.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920776' 2026-02-21T11:20:08.135 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920776 2026-02-21T11:20:08.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:20:08.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920776 -lt 141733920776 2026-02-21T11:20:08.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:08.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-176093659143 2026-02-21T11:20:08.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:08.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:20:08.357 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-176093659143 2026-02-21T11:20:08.357 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:08.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659143 2026-02-21T11:20:08.359 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 176093659143 2026-02-21T11:20:08.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 176093659143' 2026-02-21T11:20:08.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:20:08.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659144 -lt 176093659143 2026-02-21T11:20:08.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:08.583 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-214748364806 2026-02-21T11:20:08.584 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:08.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:20:08.585 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-214748364806 2026-02-21T11:20:08.585 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:08.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364806 2026-02-21T11:20:08.587 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 214748364806 2026-02-21T11:20:08.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 214748364806' 2026-02-21T11:20:08.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:20:08.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364807 -lt 214748364806 2026-02-21T11:20:08.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:20:08.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:08.811 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:20:09.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:20:09.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:20:09.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:20:09.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:20:09.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:20:09.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:20:09.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:20:09.088 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:20:09.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:20:09.297 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:20:09.297 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:09.297 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-21T11:20:09.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:20:09.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:20:09.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-21T11:20:09.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:20:09.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:20:09.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:20:09.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:20:09.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:20:09.856 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:20:09.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:20:10.138 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:20:10.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:20:10.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:20:10.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:20:10.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:20:10.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:20:10.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:20:10.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:20:10.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:20:10.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:20:10.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:20:10.461 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:20:10.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:20:10.463 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:20:10.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:20:10.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:20:10.465 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:20:10.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:20:10.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:20:10.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:20:10.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:20:10.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:20:10.469 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:20:10.469 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:20:10.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:20:10.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:20:10.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:20:10.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:20:10.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:20:10.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:20:10.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:20:10.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:20:10.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:20:10.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:20:10.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:20:10.497 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:20:10.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:20:10.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:20:10.499 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:20:10.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:20:10.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:20:10.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:20:10.501 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:20:10.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:20:10.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:20:10.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:20:10.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:20:10.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:20:10.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:20:10.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:20:10.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:20:10.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:20:10.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:20:10.507 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:20:10.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:20:10.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:20:10.531 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:20:10.531 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:10.531 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:10.531 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:10.531 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.532 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.532 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:10.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:20:10.558 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:20:10.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:20:10.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:20:10.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:20:10.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:20:10.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:20:10.627 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:20:10.627 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:20:10.627 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:20:10.627 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:20:10.627 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:20:10.627 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:10.628 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.628 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.628 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:20:10.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:20:10.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:20:10.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:20:10.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:20:10.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:20:10.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:20:10.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:20:10.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:20:10.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:20:10.789 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:20:10.790 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:10.790 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:10.790 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:10.790 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:10.790 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:10.790 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:10.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:20:10.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:20:10.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:20:10.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:20:10.929 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:20:10.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:20:11.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:20:11.941 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:20:11.941 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:20:11.941 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:11.941 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:11.941 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:11.942 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:20:11.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:20:11.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:20:11.987 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:20:11.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:20:11.995 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:20:10.542+0000 7f341fc3aec0 0 load: jerasure load: lrc 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:20:11.996 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-21T11:20:11.997 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-21T11:20:12.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:12.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:20:12.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:12.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:12.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:12.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:12.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:20:12.004 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:12.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dcb28170-79bc-4019-af7a-793189384756 2026-02-21T11:20:12.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dcb28170-79bc-4019-af7a-793189384756' 2026-02-21T11:20:12.004 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 dcb28170-79bc-4019-af7a-793189384756 2026-02-21T11:20:12.005 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:12.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDslJlp9YGWABAAYKcZeBgy8dz7qLIX1uzQEg== 2026-02-21T11:20:12.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDslJlp9YGWABAAYKcZeBgy8dz7qLIX1uzQEg=="}' 2026-02-21T11:20:12.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dcb28170-79bc-4019-af7a-793189384756 -i td/test-erasure-eio/0/new.json 2026-02-21T11:20:12.125 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:12.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:20:12.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDslJlp9YGWABAAYKcZeBgy8dz7qLIX1uzQEg== --osd-uuid dcb28170-79bc-4019-af7a-793189384756 2026-02-21T11:20:12.149 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:12.142+0000 7fc9a2423a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:12.150 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:12.142+0000 7fc9a2423a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:12.151 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:12.146+0000 7fc9a2423a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:12.152 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:12.146+0000 7fc9a2423a40 -1 bdev(0x562ef2182400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:12.152 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:12.146+0000 7fc9a2423a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:20:12.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:20:12.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:12.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:20:12.828 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:20:12.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:12.939 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:20:12.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:20:12.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:12.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:12.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:12.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:13.005 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:12.994+0000 7fa31ab17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:13.010 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:13.002+0000 7fa31ab17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:13.010 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:13.002+0000 7fa31ab17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:13.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:20:13.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:13.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:20:13.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:13.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:13.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:13.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:13.094 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:13.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:13.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:20:13.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:13.369 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:13.362+0000 7fa31ab17a40 -1 Falling back to public interface 2026-02-21T11:20:13.904 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:13.898+0000 7fa31ab17a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:20:14.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:14.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:14.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:14.200 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:14.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:14.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:20:14.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:15.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:15.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:15.420 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:15.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:15.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:15.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:20:15.634 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/694794890,v1:127.0.0.1:6803/694794890] [v2:127.0.0.1:6804/694794890,v1:127.0.0.1:6805/694794890] exists,up dcb28170-79bc-4019-af7a-793189384756 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:15.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:15.636 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:20:15.637 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:15.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=537c3f15-f135-4d0e-a680-bc9809c1bebe 2026-02-21T11:20:15.638 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 537c3f15-f135-4d0e-a680-bc9809c1bebe 2026-02-21T11:20:15.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 537c3f15-f135-4d0e-a680-bc9809c1bebe' 2026-02-21T11:20:15.639 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:15.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDvlJlpVsCkJhAAWR3m3zO1M+qnqqNKQpLe8Q== 2026-02-21T11:20:15.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDvlJlpVsCkJhAAWR3m3zO1M+qnqqNKQpLe8Q=="}' 2026-02-21T11:20:15.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 537c3f15-f135-4d0e-a680-bc9809c1bebe -i td/test-erasure-eio/1/new.json 2026-02-21T11:20:15.868 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:15.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:20:15.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDvlJlpVsCkJhAAWR3m3zO1M+qnqqNKQpLe8Q== --osd-uuid 537c3f15-f135-4d0e-a680-bc9809c1bebe 2026-02-21T11:20:15.904 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:15.898+0000 7f00be23aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:15.906 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:15.898+0000 7f00be23aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:15.907 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:15.898+0000 7f00be23aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:15.907 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:15.902+0000 7f00be23aa40 -1 bdev(0x55b520476400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:15.907 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:15.902+0000 7f00be23aa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:20:16.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:20:16.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:16.532 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:20:16.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:20:16.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:16.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:20:16.834 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:20:16.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:16.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:16.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:16.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:16.857 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:16.850+0000 7f7767a3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:16.859 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:16.850+0000 7f7767a3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:16.860 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:16.854+0000 7f7767a3fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:17.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:20:17.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:17.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:20:17.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:17.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:17.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:17.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:17.041 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:17.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:17.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:20:17.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:17.305 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:17.298+0000 7f7767a3fa40 -1 Falling back to public interface 2026-02-21T11:20:17.889 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:17.882+0000 7f7767a3fa40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:20:18.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:18.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:18.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:18.262 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:18.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:18.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:20:18.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:19.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:19.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:19.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:19.476 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:19.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:19.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:20:19.686 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3176758151,v1:127.0.0.1:6811/3176758151] [v2:127.0.0.1:6812/3176758151,v1:127.0.0.1:6813/3176758151] exists,up 537c3f15-f135-4d0e-a680-bc9809c1bebe 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:20:19.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:19.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:19.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:20:19.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:19.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56c413d5-fdef-4992-8702-926805386497 2026-02-21T11:20:19.691 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 56c413d5-fdef-4992-8702-926805386497 2026-02-21T11:20:19.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 56c413d5-fdef-4992-8702-926805386497' 2026-02-21T11:20:19.691 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:19.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDzlJlpxOO/KRAA/pcHvRTtf7MnHcKGrOfLQw== 2026-02-21T11:20:19.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDzlJlpxOO/KRAA/pcHvRTtf7MnHcKGrOfLQw=="}' 2026-02-21T11:20:19.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56c413d5-fdef-4992-8702-926805386497 -i td/test-erasure-eio/2/new.json 2026-02-21T11:20:19.930 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:19.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:20:19.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDzlJlpxOO/KRAA/pcHvRTtf7MnHcKGrOfLQw== --osd-uuid 56c413d5-fdef-4992-8702-926805386497 2026-02-21T11:20:19.955 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:19.950+0000 7f7262327a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:19.957 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:19.950+0000 7f7262327a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:19.958 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:19.950+0000 7f7262327a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:19.959 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:19.950+0000 7f7262327a40 -1 bdev(0x55c51e430400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:19.959 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:19.950+0000 7f7262327a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:20:20.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:20:20.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:20.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:20:20.572 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:20:20.572 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:20.857 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:20:20.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:20:20.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:20.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:20.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:20.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:20.881 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:20.874+0000 7fa1adff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:20.883 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:20.874+0000 7fa1adff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:20.884 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:20.878+0000 7fa1adff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:21.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:21.066 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:21.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:21.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:20:21.277 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:21.270+0000 7fa1adff7a40 -1 Falling back to public interface 2026-02-21T11:20:21.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:21.923 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:21.918+0000 7fa1adff7a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:20:22.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:22.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:22.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:22.284 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:22.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:22.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:20:22.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:23.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:23.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:23.502 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:23.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:20:23.717 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2441218507,v1:127.0.0.1:6819/2441218507] [v2:127.0.0.1:6820/2441218507,v1:127.0.0.1:6821/2441218507] exists,up 56c413d5-fdef-4992-8702-926805386497 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:23.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:23.719 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:23.720 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:23.720 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:23.720 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:23.720 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:23.721 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:20:23.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:23.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2041df15-c69c-4c55-bfb7-fde95296eade 2026-02-21T11:20:23.722 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 2041df15-c69c-4c55-bfb7-fde95296eade 2026-02-21T11:20:23.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2041df15-c69c-4c55-bfb7-fde95296eade' 2026-02-21T11:20:23.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:23.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3lJlpfJWWKxAA2pVVesK9d2tvt6RwhpxT1g== 2026-02-21T11:20:23.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3lJlpfJWWKxAA2pVVesK9d2tvt6RwhpxT1g=="}' 2026-02-21T11:20:23.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2041df15-c69c-4c55-bfb7-fde95296eade -i td/test-erasure-eio/3/new.json 2026-02-21T11:20:23.952 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:20:23.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:20:23.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3lJlpfJWWKxAA2pVVesK9d2tvt6RwhpxT1g== --osd-uuid 2041df15-c69c-4c55-bfb7-fde95296eade 2026-02-21T11:20:23.989 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:23.982+0000 7f44c560ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:23.991 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:23.982+0000 7f44c560ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:23.992 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:23.986+0000 7f44c560ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:23.993 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:23.986+0000 7f44c560ca40 -1 bdev(0x55b40d5b2400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:23.993 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:23.986+0000 7f44c560ca40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:20:24.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:20:24.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:24.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:20:24.664 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:20:24.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:24.951 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:20:24.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:20:24.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:24.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:24.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:24.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:24.967 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:24.958+0000 7efd25f5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:24.968 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:24.962+0000 7efd25f5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:24.969 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:24.962+0000 7efd25f5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:25.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:20:25.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:25.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:20:25.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:25.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:25.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:25.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:25.167 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:25.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:20:25.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:25.529 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:25.522+0000 7efd25f5aa40 -1 Falling back to public interface 2026-02-21T11:20:26.156 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:26.150+0000 7efd25f5aa40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:20:26.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:26.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:26.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:26.384 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:26.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:26.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:20:26.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:27.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:27.604 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:27.605 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:27.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:27.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:27.605 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:20:27.821 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4018532915,v1:127.0.0.1:6827/4018532915] [v2:127.0.0.1:6828/4018532915,v1:127.0.0.1:6829/4018532915] exists,up 2041df15-c69c-4c55-bfb7-fde95296eade 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:27.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:27.823 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:27.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:20:27.825 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:27.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f50ed06c-3745-4fe0-b597-14eff083ccdd 2026-02-21T11:20:27.826 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 f50ed06c-3745-4fe0-b597-14eff083ccdd 2026-02-21T11:20:27.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 f50ed06c-3745-4fe0-b597-14eff083ccdd' 2026-02-21T11:20:27.826 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:27.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD7lJlpxlDVMRAATzkoYBOlLs6WpEDwawL/mA== 2026-02-21T11:20:27.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD7lJlpxlDVMRAATzkoYBOlLs6WpEDwawL/mA=="}' 2026-02-21T11:20:27.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f50ed06c-3745-4fe0-b597-14eff083ccdd -i td/test-erasure-eio/4/new.json 2026-02-21T11:20:28.064 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:20:28.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:20:28.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD7lJlpxlDVMRAATzkoYBOlLs6WpEDwawL/mA== --osd-uuid f50ed06c-3745-4fe0-b597-14eff083ccdd 2026-02-21T11:20:28.092 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:28.086+0000 7fafb307aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:28.093 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:28.086+0000 7fafb307aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:28.094 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:28.086+0000 7fafb307aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:28.094 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:28.086+0000 7fafb307aa40 -1 bdev(0x556659c40400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:28.095 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:28.086+0000 7fafb307aa40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:20:28.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:20:28.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:28.727 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:20:28.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:20:28.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:29.016 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:20:29.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:20:29.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:29.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:29.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:29.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:29.040 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:29.034+0000 7f6d55aeda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:29.042 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:29.034+0000 7f6d55aeda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:29.043 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:29.038+0000 7f6d55aeda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:29.222 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:29.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:29.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:20:29.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:29.489 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:29.482+0000 7f6d55aeda40 -1 Falling back to public interface 2026-02-21T11:20:30.040 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:30.034+0000 7f6d55aeda40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:20:30.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:30.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:30.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:30.439 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:30.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:30.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:20:30.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:31.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:31.658 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:31.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:31.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:31.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:31.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:20:31.869 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 36 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1406542904,v1:127.0.0.1:6835/1406542904] [v2:127.0.0.1:6836/1406542904,v1:127.0.0.1:6837/1406542904] exists,up f50ed06c-3745-4fe0-b597-14eff083ccdd 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:31.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-21T11:20:31.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:31.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b4cd33eb-339a-4c40-a283-316b3b291ff8 2026-02-21T11:20:31.874 INFO:tasks.workunit.client.0.trial166.stdout:add osd5 b4cd33eb-339a-4c40-a283-316b3b291ff8 2026-02-21T11:20:31.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 b4cd33eb-339a-4c40-a283-316b3b291ff8' 2026-02-21T11:20:31.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:31.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD/lJlp3EqvNBAADt5qYhSLAANMALwZEI8dJQ== 2026-02-21T11:20:31.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD/lJlp3EqvNBAADt5qYhSLAANMALwZEI8dJQ=="}' 2026-02-21T11:20:31.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b4cd33eb-339a-4c40-a283-316b3b291ff8 -i td/test-erasure-eio/5/new.json 2026-02-21T11:20:32.111 INFO:tasks.workunit.client.0.trial166.stdout:5 2026-02-21T11:20:32.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-21T11:20:32.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD/lJlp3EqvNBAADt5qYhSLAANMALwZEI8dJQ== --osd-uuid b4cd33eb-339a-4c40-a283-316b3b291ff8 2026-02-21T11:20:32.137 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:32.130+0000 7f0efb69fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:32.139 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:32.130+0000 7f0efb69fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:32.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:32.134+0000 7f0efb69fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:32.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:32.134+0000 7f0efb69fa40 -1 bdev(0x5581ae0d2400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:32.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:32.134+0000 7f0efb69fa40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-21T11:20:32.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-21T11:20:32.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:32.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-21T11:20:32.811 INFO:tasks.workunit.client.0.trial166.stdout:adding osd5 key to auth repository 2026-02-21T11:20:32.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:33.105 INFO:tasks.workunit.client.0.trial166.stdout:start osd.5 2026-02-21T11:20:33.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-21T11:20:33.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:33.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:33.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:33.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:33.129 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:33.122+0000 7fe4354e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:33.131 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:33.126+0000 7fe4354e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:33.132 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:33.126+0000 7fe4354e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:33.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:33.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:20:33.530 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:33.537 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:33.530+0000 7fe4354e0a40 -1 Falling back to public interface 2026-02-21T11:20:34.282 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:34.274+0000 7fe4354e0a40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:20:34.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:34.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:34.532 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:34.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:34.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:34.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:20:34.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:35.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:35.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:35.768 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:35.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:35.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:35.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:20:35.984 INFO:tasks.workunit.client.0.trial166.stdout:osd.5 up in weight 1 up_from 46 up_thru 47 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2146175667,v1:127.0.0.1:6843/2146175667] [v2:127.0.0.1:6844/2146175667,v1:127.0.0.1:6845/2146175667] exists,up b4cd33eb-339a-4c40-a283-316b3b291ff8 2026-02-21T11:20:35.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:35.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:35.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:35.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-21T11:20:35.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:35.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=010ff347-66ac-49ff-88f9-61b8663395de 2026-02-21T11:20:35.992 INFO:tasks.workunit.client.0.trial166.stdout:add osd6 010ff347-66ac-49ff-88f9-61b8663395de 2026-02-21T11:20:35.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 010ff347-66ac-49ff-88f9-61b8663395de' 2026-02-21T11:20:35.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:36.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQADlZlpgV+EOxAAl19GvljKy++yQbTy5wcevw== 2026-02-21T11:20:36.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQADlZlpgV+EOxAAl19GvljKy++yQbTy5wcevw=="}' 2026-02-21T11:20:36.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 010ff347-66ac-49ff-88f9-61b8663395de -i td/test-erasure-eio/6/new.json 2026-02-21T11:20:36.217 INFO:tasks.workunit.client.0.trial166.stdout:6 2026-02-21T11:20:36.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-21T11:20:36.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQADlZlpgV+EOxAAl19GvljKy++yQbTy5wcevw== --osd-uuid 010ff347-66ac-49ff-88f9-61b8663395de 2026-02-21T11:20:36.254 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:36.246+0000 7fdf4c40da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:36.255 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:36.246+0000 7fdf4c40da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:36.256 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:36.250+0000 7fdf4c40da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:36.256 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:36.250+0000 7fdf4c40da40 -1 bdev(0x55671e560400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:36.257 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:36.250+0000 7fdf4c40da40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-21T11:20:36.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-21T11:20:36.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:36.932 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-21T11:20:36.932 INFO:tasks.workunit.client.0.trial166.stdout:adding osd6 key to auth repository 2026-02-21T11:20:36.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:37.222 INFO:tasks.workunit.client.0.trial166.stdout:start osd.6 2026-02-21T11:20:37.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-21T11:20:37.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:37.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:37.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:37.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:37.246 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:37.238+0000 7f21cb3a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:37.248 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:37.242+0000 7f21cb3a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:37.249 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:37.242+0000 7f21cb3a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-21T11:20:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:20:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:37.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:37.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:37.432 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:37.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:37.433 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:20:37.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:37.653 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:37.646+0000 7f21cb3a5a40 -1 Falling back to public interface 2026-02-21T11:20:38.292 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:38.286+0000 7f21cb3a5a40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:20:38.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:38.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:38.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:38.644 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:38.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:38.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:20:38.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:39.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:39.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:39.850 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:39.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:39.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:39.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:20:40.067 INFO:tasks.workunit.client.0.trial166.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/727149419,v1:127.0.0.1:6851/727149419] [v2:127.0.0.1:6852/727149419,v1:127.0.0.1:6853/727149419] exists,up 010ff347-66ac-49ff-88f9-61b8663395de 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:40.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:40.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-21T11:20:40.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:40.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=de33aa73-9700-4d09-9b71-9e10cfb0e1c4 2026-02-21T11:20:40.072 INFO:tasks.workunit.client.0.trial166.stdout:add osd7 de33aa73-9700-4d09-9b71-9e10cfb0e1c4 2026-02-21T11:20:40.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 de33aa73-9700-4d09-9b71-9e10cfb0e1c4' 2026-02-21T11:20:40.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:40.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAIlZlpQSzrBBAAqSKHCi8P8X4K4CWnMjYEpw== 2026-02-21T11:20:40.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAIlZlpQSzrBBAAqSKHCi8P8X4K4CWnMjYEpw=="}' 2026-02-21T11:20:40.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new de33aa73-9700-4d09-9b71-9e10cfb0e1c4 -i td/test-erasure-eio/7/new.json 2026-02-21T11:20:40.298 INFO:tasks.workunit.client.0.trial166.stdout:7 2026-02-21T11:20:40.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-21T11:20:40.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAIlZlpQSzrBBAAqSKHCi8P8X4K4CWnMjYEpw== --osd-uuid de33aa73-9700-4d09-9b71-9e10cfb0e1c4 2026-02-21T11:20:40.334 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:40.326+0000 7f26387d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:40.336 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:40.330+0000 7f26387d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:40.337 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:40.330+0000 7f26387d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:40.337 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:40.330+0000 7f26387d1a40 -1 bdev(0x55c88312c400 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:40.338 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:40.330+0000 7f26387d1a40 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-21T11:20:40.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-21T11:20:40.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:40.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-21T11:20:40.991 INFO:tasks.workunit.client.0.trial166.stdout:adding osd7 key to auth repository 2026-02-21T11:20:40.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:41.277 INFO:tasks.workunit.client.0.trial166.stdout:start osd.7 2026-02-21T11:20:41.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-21T11:20:41.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:41.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:41.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:41.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:41.293 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:41.286+0000 7f20988f4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:41.295 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:41.290+0000 7f20988f4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:41.296 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:41.290+0000 7f20988f4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:41.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-21T11:20:41.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:41.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-21T11:20:41.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:41.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:41.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:41.481 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:41.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:41.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:41.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-21T11:20:41.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:41.700 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:41.694+0000 7f20988f4a40 -1 Falling back to public interface 2026-02-21T11:20:42.368 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:42.362+0000 7f20988f4a40 -1 osd.7 0 log_to_monitors true 2026-02-21T11:20:42.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:42.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:42.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:42.699 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:42.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:42.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-21T11:20:42.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:43.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:43.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:43.913 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:43.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:43.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:43.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-21T11:20:44.126 INFO:tasks.workunit.client.0.trial166.stdout:osd.7 up in weight 1 up_from 60 up_thru 60 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/3094994739,v1:127.0.0.1:6859/3094994739] [v2:127.0.0.1:6860/3094994739,v1:127.0.0.1:6861/3094994739] exists,up de33aa73-9700-4d09-9b71-9e10cfb0e1c4 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:20:44.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:20:44.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-21T11:20:44.129 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:20:44.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=70dd3c3a-3d79-463f-9aed-77064c120fb0 2026-02-21T11:20:44.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 70dd3c3a-3d79-463f-9aed-77064c120fb0' 2026-02-21T11:20:44.130 INFO:tasks.workunit.client.0.trial166.stdout:add osd8 70dd3c3a-3d79-463f-9aed-77064c120fb0 2026-02-21T11:20:44.130 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:20:44.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAMlZlpvUJeCBAAz1yOL4YX0U8gx7Ucu7vOqw== 2026-02-21T11:20:44.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAMlZlpvUJeCBAAz1yOL4YX0U8gx7Ucu7vOqw=="}' 2026-02-21T11:20:44.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 70dd3c3a-3d79-463f-9aed-77064c120fb0 -i td/test-erasure-eio/8/new.json 2026-02-21T11:20:44.370 INFO:tasks.workunit.client.0.trial166.stdout:8 2026-02-21T11:20:44.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-21T11:20:44.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMlZlpvUJeCBAAz1yOL4YX0U8gx7Ucu7vOqw== --osd-uuid 70dd3c3a-3d79-463f-9aed-77064c120fb0 2026-02-21T11:20:44.405 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:44.398+0000 7fe0f5bb4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:44.407 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:44.398+0000 7fe0f5bb4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:44.408 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:44.402+0000 7fe0f5bb4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:44.408 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:44.402+0000 7fe0f5bb4a40 -1 bdev(0x5582d0232400 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-21T11:20:44.409 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:44.402+0000 7fe0f5bb4a40 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-21T11:20:45.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-21T11:20:45.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:20:45.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-21T11:20:45.056 INFO:tasks.workunit.client.0.trial166.stdout:adding osd8 key to auth repository 2026-02-21T11:20:45.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:20:45.351 INFO:tasks.workunit.client.0.trial166.stdout:start osd.8 2026-02-21T11:20:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-21T11:20:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:20:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:20:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:20:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:20:45.375 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:45.370+0000 7f562acc0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:45.377 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:45.370+0000 7f562acc0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:45.378 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:45.370+0000 7f562acc0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:20:45.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-21T11:20:45.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:20:45.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-21T11:20:45.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:20:45.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:20:45.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:45.570 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:20:45.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:20:45.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:45.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-21T11:20:45.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:45.762+0000 7f562acc0a40 -1 Falling back to public interface 2026-02-21T11:20:45.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:46.368 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:20:46.362+0000 7f562acc0a40 -1 osd.8 0 log_to_monitors true 2026-02-21T11:20:46.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:46.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:46.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:20:46.786 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:20:46.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:46.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-21T11:20:47.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:20:48.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:20:48.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:20:48.007 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:20:48.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:20:48.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:20:48.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-21T11:20:48.228 INFO:tasks.workunit.client.0.trial166.stdout:osd.8 up in weight 1 up_from 68 up_thru 71 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/863135251,v1:127.0.0.1:6867/863135251] [v2:127.0.0.1:6868/863135251,v1:127.0.0.1:6869/863135251] exists,up 70dd3c3a-3d79-463f-9aed-77064c120fb0 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:48.229 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:48.230 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:20:48.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:20:48.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:20:48.294 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:20:48.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:20:48.302 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:20:13.366+0000 7fa31ab17a40 0 load: jerasure load: lrc 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:20:48.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-21T11:20:48.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:20:48.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:20:48.946 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:20:48.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:20:49.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:20:49.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:20:49.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:20:49.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:20:49.957 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:20:49.957 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:20:49.958 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:20:49.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:20:49.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:20:49.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:20:50.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:20:50.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:20:50.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:20:50.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:20:50.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:20:50.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:8' 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:20:50.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.279 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:20:50.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803786 2026-02-21T11:20:50.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803786 2026-02-21T11:20:50.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786' 2026-02-21T11:20:50.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.357 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:20:50.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574857 2026-02-21T11:20:50.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574857 2026-02-21T11:20:50.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857' 2026-02-21T11:20:50.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:20:50.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411336 2026-02-21T11:20:50.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411336 2026-02-21T11:20:50.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336' 2026-02-21T11:20:50.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:20:50.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116999 2026-02-21T11:20:50.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116999 2026-02-21T11:20:50.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-115964116999' 2026-02-21T11:20:50.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.594 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:20:50.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822662 2026-02-21T11:20:50.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822662 2026-02-21T11:20:50.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-115964116999 4-154618822662' 2026-02-21T11:20:50.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.673 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:20:50.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=197568495622 2026-02-21T11:20:50.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 197568495622 2026-02-21T11:20:50.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-115964116999 4-154618822662 5-197568495622' 2026-02-21T11:20:50.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.753 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:20:50.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=223338299397 2026-02-21T11:20:50.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 223338299397 2026-02-21T11:20:50.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-115964116999 4-154618822662 5-197568495622 6-223338299397' 2026-02-21T11:20:50.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.830 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:20:50.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=257698037764 2026-02-21T11:20:50.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 257698037764 2026-02-21T11:20:50.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-115964116999 4-154618822662 5-197568495622 6-223338299397 7-257698037764' 2026-02-21T11:20:50.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:50.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:20:50.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776132 2026-02-21T11:20:50.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776132 2026-02-21T11:20:50.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-115964116999 4-154618822662 5-197568495622 6-223338299397 7-257698037764 8-292057776132' 2026-02-21T11:20:50.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:50.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803786 2026-02-21T11:20:50.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:50.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:20:50.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803786 2026-02-21T11:20:50.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:50.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803786 2026-02-21T11:20:50.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-21T11:20:50.992 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803786 2026-02-21T11:20:50.992 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:20:51.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-21T11:20:51.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:51.206 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574857 2026-02-21T11:20:51.207 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:51.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:20:51.208 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574857 2026-02-21T11:20:51.209 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:51.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574857 2026-02-21T11:20:51.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-21T11:20:51.210 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574857 2026-02-21T11:20:51.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:20:51.433 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-21T11:20:51.433 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:51.434 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411336 2026-02-21T11:20:51.434 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:51.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:20:51.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411336 2026-02-21T11:20:51.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:51.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411336 2026-02-21T11:20:51.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-21T11:20:51.438 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 77309411336 2026-02-21T11:20:51.438 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:20:51.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-21T11:20:51.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:51.649 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116999 2026-02-21T11:20:51.649 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:51.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:20:51.651 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116999 2026-02-21T11:20:51.651 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:51.652 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116999 2026-02-21T11:20:51.652 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 115964116999 2026-02-21T11:20:51.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116999' 2026-02-21T11:20:51.653 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:20:51.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116999 -lt 115964116999 2026-02-21T11:20:51.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:51.866 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822662 2026-02-21T11:20:51.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:51.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:20:51.868 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822662 2026-02-21T11:20:51.869 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:51.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822662 2026-02-21T11:20:51.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822662' 2026-02-21T11:20:51.870 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 154618822662 2026-02-21T11:20:51.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:20:52.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822662 -lt 154618822662 2026-02-21T11:20:52.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:52.096 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-197568495622 2026-02-21T11:20:52.096 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:52.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:20:52.098 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-197568495622 2026-02-21T11:20:52.098 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:52.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=197568495622 2026-02-21T11:20:52.100 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 197568495622 2026-02-21T11:20:52.101 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 197568495622' 2026-02-21T11:20:52.101 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:20:52.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 197568495622 -lt 197568495622 2026-02-21T11:20:52.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:52.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-223338299397 2026-02-21T11:20:52.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:52.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:20:52.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-223338299397 2026-02-21T11:20:52.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:52.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=223338299397 2026-02-21T11:20:52.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 223338299397' 2026-02-21T11:20:52.320 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 223338299397 2026-02-21T11:20:52.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:20:52.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 223338299397 -lt 223338299397 2026-02-21T11:20:52.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:52.550 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-257698037764 2026-02-21T11:20:52.550 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:52.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-21T11:20:52.551 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-257698037764 2026-02-21T11:20:52.552 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:52.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=257698037764 2026-02-21T11:20:52.553 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.7 seq 257698037764 2026-02-21T11:20:52.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 257698037764' 2026-02-21T11:20:52.554 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-21T11:20:52.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 257698037764 -lt 257698037764 2026-02-21T11:20:52.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:52.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-292057776132 2026-02-21T11:20:52.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:52.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:20:52.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-292057776132 2026-02-21T11:20:52.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:52.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776132 2026-02-21T11:20:52.782 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 292057776132 2026-02-21T11:20:52.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 292057776132' 2026-02-21T11:20:52.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:20:52.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776132 -lt 292057776132 2026-02-21T11:20:52.998 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:20:52.998 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:52.998 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:20:53.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:20:53.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:20:53.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:20:53.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:53.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:20:53.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:53.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:20:53.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:53.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:20:53.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:20:53.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:20:53.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-21T11:20:53.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-21T11:20:53.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:20:53.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:20:53.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:20:53.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:20:53.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:20:53.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:20:53.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-21T11:20:53.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:20:53.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:20:53.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:20:53.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:20:53.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:20:53.801 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-21T11:20:53.802 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:20:53.802 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:20:53.802 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:20:53.802 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:20:54.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:20:54.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:20:54.016 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:20:54.016 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:20:54.016 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:20:54.016 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:20:54.017 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:54.017 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:54.017 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:54.017 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:20:54.017 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:20:54.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:20:54.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:54.079 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:54.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:20:54.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:20:54.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-21T11:20:54.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-21T11:20:54.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:20:54.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:20:54.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:20:54.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=3 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:20:54.141 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:20:54.142 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:20:54.142 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:20:54.357 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-21T11:20:54.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-21T11:20:54.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:20:54.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=0 2026-02-21T11:20:54.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:20:54.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/0/type 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:20:54.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:20:54.360 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-21T11:20:54.360 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:20:54.360 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:20:54.360 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:54.360 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:54.360 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:54.361 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:20:54.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:20:54.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:20:54.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:20:54.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:20:54.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.0 2026-02-21T11:20:54.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:20:54.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:20:54.424 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:54.424 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:54.424 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:54.424 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:20:54.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:20:54.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-21T11:20:54.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:20:54.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=4 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:20:54.486 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:20:54.487 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:20:54.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:20:54.707 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:54.707 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:54.707 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:20:54.707 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:20:54.708 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:54.708 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:54.708 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:20:54.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-21T11:20:54.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-21T11:20:54.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=6 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/6/type 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:20:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.6.asok 2026-02-21T11:20:54.710 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:20:54.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:20:54.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.6 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:20:54.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:20:54.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.6.asok 2026-02-21T11:20:54.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:20:54.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-21T11:20:54.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-21T11:20:54.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:20:54.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:20:54.832 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:20:54.832 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:20:55.043 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:20:55.043 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:55.043 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:55.043 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:20:55.043 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:20:55.043 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr:7' 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:20:55.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:20:55.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:20:55.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:20:55.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:20:55.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:20:55.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-21T11:20:55.556 INFO:tasks.workunit.client.0.trial166.stderr:osd.7 is already down. 2026-02-21T11:20:55.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-21T11:20:55.813 INFO:tasks.workunit.client.0.trial166.stderr:osd.7 is already out. 2026-02-21T11:20:55.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-21T11:20:55.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:20:55.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:20:55.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:20:55.824 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:20:55.824 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:20:55.824 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:20:55.825 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:20:55.825 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:20:55.825 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:20:55.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:20:55.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:20:55.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:20:55.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:20:55.932 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:20:55.932 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:20:56.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:6 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:7 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:8' 2026-02-21T11:20:56.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:20:56.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.145 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:20:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803789 2026-02-21T11:20:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803789 2026-02-21T11:20:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789' 2026-02-21T11:20:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:20:56.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574860 2026-02-21T11:20:56.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574860 2026-02-21T11:20:56.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860' 2026-02-21T11:20:56.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:20:56.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411339 2026-02-21T11:20:56.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411339 2026-02-21T11:20:56.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860 2-77309411339' 2026-02-21T11:20:56.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:20:56.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117002 2026-02-21T11:20:56.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117002 2026-02-21T11:20:56.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860 2-77309411339 3-115964117002' 2026-02-21T11:20:56.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.463 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:20:56.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822666 2026-02-21T11:20:56.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822666 2026-02-21T11:20:56.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860 2-77309411339 3-115964117002 4-154618822666' 2026-02-21T11:20:56.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.543 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:20:56.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=197568495625 2026-02-21T11:20:56.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 197568495625 2026-02-21T11:20:56.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860 2-77309411339 3-115964117002 4-154618822666 5-197568495625' 2026-02-21T11:20:56.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.619 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:20:56.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=223338299400 2026-02-21T11:20:56.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 223338299400 2026-02-21T11:20:56.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860 2-77309411339 3-115964117002 4-154618822666 5-197568495625 6-223338299400' 2026-02-21T11:20:56.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.697 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-21T11:20:56.758 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-21T11:20:56.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-21T11:20:56.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-21T11:20:56.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-21T11:20:56.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:20:56.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-21T11:20:56.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776135 2026-02-21T11:20:56.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776135 2026-02-21T11:20:56.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-55834574860 2-77309411339 3-115964117002 4-154618822666 5-197568495625 6-223338299400 8-292057776135' 2026-02-21T11:20:56.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:56.834 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803789 2026-02-21T11:20:56.834 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:56.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:20:56.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803789 2026-02-21T11:20:56.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:56.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803789 2026-02-21T11:20:56.837 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803789 2026-02-21T11:20:56.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-21T11:20:56.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:20:57.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-21T11:20:57.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:20:58.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:20:58.061 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:20:58.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-21T11:20:58.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:58.271 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574860 2026-02-21T11:20:58.271 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:58.272 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:20:58.273 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574860 2026-02-21T11:20:58.273 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:58.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574860 2026-02-21T11:20:58.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574860' 2026-02-21T11:20:58.275 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574860 2026-02-21T11:20:58.275 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:20:58.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574860 -lt 55834574860 2026-02-21T11:20:58.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:58.484 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411339 2026-02-21T11:20:58.484 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:58.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:20:58.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411339 2026-02-21T11:20:58.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:58.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411339 2026-02-21T11:20:58.488 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 77309411339 2026-02-21T11:20:58.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411339' 2026-02-21T11:20:58.488 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:20:58.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411339 -lt 77309411339 2026-02-21T11:20:58.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:58.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117002 2026-02-21T11:20:58.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:58.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:20:58.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117002 2026-02-21T11:20:58.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:58.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117002 2026-02-21T11:20:58.711 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 115964117002 2026-02-21T11:20:58.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117002' 2026-02-21T11:20:58.711 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:20:58.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117002 -lt 115964117002 2026-02-21T11:20:58.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:58.938 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822666 2026-02-21T11:20:58.938 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:58.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:20:58.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822666 2026-02-21T11:20:58.941 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:58.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822666 2026-02-21T11:20:58.942 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 154618822666 2026-02-21T11:20:58.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822666' 2026-02-21T11:20:58.942 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:20:59.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822666 -lt 154618822666 2026-02-21T11:20:59.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:59.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-197568495625 2026-02-21T11:20:59.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:59.166 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:20:59.166 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-197568495625 2026-02-21T11:20:59.167 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:59.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=197568495625 2026-02-21T11:20:59.168 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 197568495625 2026-02-21T11:20:59.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 197568495625' 2026-02-21T11:20:59.169 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:20:59.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 197568495625 -lt 197568495625 2026-02-21T11:20:59.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:59.377 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-223338299400 2026-02-21T11:20:59.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:59.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:20:59.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-223338299400 2026-02-21T11:20:59.380 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:59.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=223338299400 2026-02-21T11:20:59.381 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 223338299400 2026-02-21T11:20:59.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 223338299400' 2026-02-21T11:20:59.382 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:20:59.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 223338299400 -lt 223338299400 2026-02-21T11:20:59.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:20:59.608 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-292057776135 2026-02-21T11:20:59.608 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:20:59.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-21T11:20:59.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-292057776135 2026-02-21T11:20:59.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:20:59.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776135 2026-02-21T11:20:59.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 292057776135' 2026-02-21T11:20:59.612 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.8 seq 292057776135 2026-02-21T11:20:59.612 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-21T11:20:59.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776136 -lt 292057776135 2026-02-21T11:20:59.820 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:20:59.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:20:59.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:21:00.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:21:00.108 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:21:00.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:21:00.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:21:00.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:21:00.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:21:00.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:21:00.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-21T11:21:00.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:21:00.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:21:00.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:21:00.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:21:00.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:21:00.876 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:21:00.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:21:01.153 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:21:01.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:21:01.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:21:01.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:21:01.478 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:21:01.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:21:01.480 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:21:01.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:21:01.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:21:01.482 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:21:01.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:01.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:21:01.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:21:01.484 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:01.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:21:01.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:21:01.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:21:01.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:21:01.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:21:01.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:21:01.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:21:01.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:21:01.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:21:01.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:21:01.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:21:01.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:21:01.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:21:01.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:21:01.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:21:01.518 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:21:01.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:21:01.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:21:01.520 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:21:01.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:01.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:21:01.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:21:01.522 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:01.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:21:01.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:21:01.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:21:01.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:21:01.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:21:01.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:21:01.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:21:01.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:21:01.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:21:01.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:21:01.530 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:21:01.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:21:01.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:21:01.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:21:01.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:21:01.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:21:01.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:21:01.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:21:01.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:21:01.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:21:01.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:21:01.553 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:21:01.553 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:01.554 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:01.554 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:01.554 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.554 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.554 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:01.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:21:01.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:21:01.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:21:01.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:21:01.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:21:01.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:21:01.583 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:21:01.584 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:21:01.585 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:21:01.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:21:01.656 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:21:01.657 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:01.657 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.657 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.658 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:21:01.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:21:01.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:21:01.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:21:01.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:21:01.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:21:01.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:21:01.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:21:01.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:21:01.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:21:01.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:21:01.830 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:01.830 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:01.830 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:01.830 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:01.830 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:01.830 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:01.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:21:01.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:21:01.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:21:01.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:21:01.951 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:21:01.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:02.962 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:21:02.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:21:02.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:21:03.013 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:21:03.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:21:03.020 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:21:01.569+0000 7f20d2ad0ec0 0 load: jerasure load: lrc 2026-02-21T11:21:03.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-21T11:21:03.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:21:03.022 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-21T11:21:03.026 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-21T11:21:03.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:03.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:21:03.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:03.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:03.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:03.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:03.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:21:03.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:03.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=995caf61-3573-4591-bb10-6426aceef3ef 2026-02-21T11:21:03.030 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 995caf61-3573-4591-bb10-6426aceef3ef 2026-02-21T11:21:03.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 995caf61-3573-4591-bb10-6426aceef3ef' 2026-02-21T11:21:03.031 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:03.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAflZlprKlFAhAAVXrDqKGqXDhTwERZJa+7VQ== 2026-02-21T11:21:03.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAflZlprKlFAhAAVXrDqKGqXDhTwERZJa+7VQ=="}' 2026-02-21T11:21:03.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 995caf61-3573-4591-bb10-6426aceef3ef -i td/test-erasure-eio/0/new.json 2026-02-21T11:21:03.156 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:03.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:21:03.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAflZlprKlFAhAAVXrDqKGqXDhTwERZJa+7VQ== --osd-uuid 995caf61-3573-4591-bb10-6426aceef3ef 2026-02-21T11:21:03.183 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:03.181+0000 7f6dd79a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:03.186 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:03.181+0000 7f6dd79a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:03.187 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:03.185+0000 7f6dd79a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:03.187 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:03.185+0000 7f6dd79a6a40 -1 bdev(0x55f352d38400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:03.188 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:03.185+0000 7f6dd79a6a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:21:03.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:21:03.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:03.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:21:03.918 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:21:03.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:04.068 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:21:04.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:21:04.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:04.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:04.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:04.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:04.085 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:04.081+0000 7fdd2d2d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:04.086 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:04.081+0000 7fdd2d2d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:04.087 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:04.085+0000 7fdd2d2d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:04.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:21:04.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:04.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:21:04.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:04.529 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:04.525+0000 7fdd2d2d0a40 -1 Falling back to public interface 2026-02-21T11:21:05.145 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:05.141+0000 7fdd2d2d0a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:21:05.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:05.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:05.402 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:05.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:05.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:05.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:21:05.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:06.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:06.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:06.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:21:06.622 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:06.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:06.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:21:06.833 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/567767313,v1:127.0.0.1:6803/567767313] [v2:127.0.0.1:6804/567767313,v1:127.0.0.1:6805/567767313] exists,up 995caf61-3573-4591-bb10-6426aceef3ef 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:06.834 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:06.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:06.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:21:06.837 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:06.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=78f827f9-276f-4509-8837-754a2f34bc25 2026-02-21T11:21:06.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 78f827f9-276f-4509-8837-754a2f34bc25' 2026-02-21T11:21:06.838 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 78f827f9-276f-4509-8837-754a2f34bc25 2026-02-21T11:21:06.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:06.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAilZlpUPSgMhAADBrhZK4SHm+h2qOXOO15YQ== 2026-02-21T11:21:06.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAilZlpUPSgMhAADBrhZK4SHm+h2qOXOO15YQ=="}' 2026-02-21T11:21:06.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 78f827f9-276f-4509-8837-754a2f34bc25 -i td/test-erasure-eio/1/new.json 2026-02-21T11:21:07.065 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:07.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:21:07.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAilZlpUPSgMhAADBrhZK4SHm+h2qOXOO15YQ== --osd-uuid 78f827f9-276f-4509-8837-754a2f34bc25 2026-02-21T11:21:07.094 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:07.089+0000 7f2dcc1baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:07.096 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:07.093+0000 7f2dcc1baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:07.097 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:07.093+0000 7f2dcc1baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:07.097 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:07.093+0000 7f2dcc1baa40 -1 bdev(0x556cdfdc8400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:07.097 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:07.093+0000 7f2dcc1baa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:21:07.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:21:07.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:07.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:21:07.727 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:21:07.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:08.006 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:21:08.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:21:08.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:08.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:08.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:08.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:08.033 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:08.029+0000 7f79284d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:08.035 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:08.029+0000 7f79284d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:08.036 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:08.033+0000 7f79284d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:08.221 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:08.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:08.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:21:08.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:08.469 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:08.465+0000 7f79284d7a40 -1 Falling back to public interface 2026-02-21T11:21:09.124 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:09.121+0000 7f79284d7a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:21:09.433 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:09.433 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:09.433 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:09.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:09.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:09.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:21:09.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:10.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:10.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:10.647 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:10.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:21:10.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:10.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:21:10.859 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1680776210,v1:127.0.0.1:6811/1680776210] [v2:127.0.0.1:6812/1680776210,v1:127.0.0.1:6813/1680776210] exists,up 78f827f9-276f-4509-8837-754a2f34bc25 2026-02-21T11:21:10.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:10.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:10.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:10.861 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:10.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:10.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:21:10.863 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:10.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1708e4e7-019b-4533-b2b0-392dfcc289dd 2026-02-21T11:21:10.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1708e4e7-019b-4533-b2b0-392dfcc289dd' 2026-02-21T11:21:10.865 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 1708e4e7-019b-4533-b2b0-392dfcc289dd 2026-02-21T11:21:10.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:10.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAmlZlpiM0+NBAAqjMJ0vcAc5jzOlXCnGjhBQ== 2026-02-21T11:21:10.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAmlZlpiM0+NBAAqjMJ0vcAc5jzOlXCnGjhBQ=="}' 2026-02-21T11:21:10.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1708e4e7-019b-4533-b2b0-392dfcc289dd -i td/test-erasure-eio/2/new.json 2026-02-21T11:21:11.101 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:11.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:21:11.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmlZlpiM0+NBAAqjMJ0vcAc5jzOlXCnGjhBQ== --osd-uuid 1708e4e7-019b-4533-b2b0-392dfcc289dd 2026-02-21T11:21:11.138 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:11.133+0000 7f4e52a2da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:11.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:11.137+0000 7f4e52a2da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:11.141 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:11.137+0000 7f4e52a2da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:11.141 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:11.137+0000 7f4e52a2da40 -1 bdev(0x557796c1e400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:11.142 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:11.137+0000 7f4e52a2da40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:21:11.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:21:11.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:11.764 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:21:11.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:21:11.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:12.059 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:21:12.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:21:12.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:12.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:12.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:12.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:12.087 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:12.081+0000 7fd6c41b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:12.089 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:12.085+0000 7fd6c41b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:12.090 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:12.085+0000 7fd6c41b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:12.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:21:12.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:12.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:21:12.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:12.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:12.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:12.279 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:12.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:12.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:12.279 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:21:12.441 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:12.437+0000 7fd6c41b3a40 -1 Falling back to public interface 2026-02-21T11:21:12.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:13.020 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:13.017+0000 7fd6c41b3a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:21:13.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:13.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:13.495 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:13.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:13.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:13.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:21:13.712 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1215481431,v1:127.0.0.1:6819/1215481431] [v2:127.0.0.1:6820/1215481431,v1:127.0.0.1:6821/1215481431] exists,up 1708e4e7-019b-4533-b2b0-392dfcc289dd 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:13.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:13.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:21:13.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:13.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e6fbcc7-2ed8-4aa5-9c65-f20fa697db7b 2026-02-21T11:21:13.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 4e6fbcc7-2ed8-4aa5-9c65-f20fa697db7b' 2026-02-21T11:21:13.717 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 4e6fbcc7-2ed8-4aa5-9c65-f20fa697db7b 2026-02-21T11:21:13.717 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:13.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAplZlpgzFwKxAA6v+01amIoZk56RUqIuLpog== 2026-02-21T11:21:13.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAplZlpgzFwKxAA6v+01amIoZk56RUqIuLpog=="}' 2026-02-21T11:21:13.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e6fbcc7-2ed8-4aa5-9c65-f20fa697db7b -i td/test-erasure-eio/3/new.json 2026-02-21T11:21:13.945 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:21:13.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:21:13.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAplZlpgzFwKxAA6v+01amIoZk56RUqIuLpog== --osd-uuid 4e6fbcc7-2ed8-4aa5-9c65-f20fa697db7b 2026-02-21T11:21:13.983 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:13.977+0000 7f6bcc202a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:13.984 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:13.981+0000 7f6bcc202a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:13.985 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:13.981+0000 7f6bcc202a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:13.986 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:13.981+0000 7f6bcc202a40 -1 bdev(0x555f26216400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:13.986 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:13.981+0000 7f6bcc202a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:21:14.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:21:14.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:14.656 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:21:14.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:21:14.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:14.949 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:21:14.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:21:14.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:14.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:14.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:14.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:14.976 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:14.973+0000 7f17ebf1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:14.978 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:14.973+0000 7f17ebf1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:14.979 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:14.977+0000 7f17ebf1ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:15.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:15.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:21:15.353 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:15.349+0000 7f17ebf1ea40 -1 Falling back to public interface 2026-02-21T11:21:15.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:15.968 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:15.965+0000 7f17ebf1ea40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:21:16.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:16.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:16.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:16.384 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:16.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:16.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:21:16.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:17.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:17.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:17.602 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:17.602 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:21:17.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:17.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:21:17.813 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 26 up_thru 29 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/239560696,v1:127.0.0.1:6827/239560696] [v2:127.0.0.1:6828/239560696,v1:127.0.0.1:6829/239560696] exists,up 4e6fbcc7-2ed8-4aa5-9c65-f20fa697db7b 2026-02-21T11:21:17.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:17.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:17.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:17.818 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:17.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:17.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:17.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:21:17.820 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:17.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8ed637ea-cc59-462d-be93-ce31ec32ba2c 2026-02-21T11:21:17.821 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 8ed637ea-cc59-462d-be93-ce31ec32ba2c 2026-02-21T11:21:17.821 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 8ed637ea-cc59-462d-be93-ce31ec32ba2c' 2026-02-21T11:21:17.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:17.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAtlZlp/S1oMRAAFEv26ssE3izUAHR85dPb+A== 2026-02-21T11:21:17.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAtlZlp/S1oMRAAFEv26ssE3izUAHR85dPb+A=="}' 2026-02-21T11:21:17.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8ed637ea-cc59-462d-be93-ce31ec32ba2c -i td/test-erasure-eio/4/new.json 2026-02-21T11:21:18.041 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:21:18.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:21:18.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAtlZlp/S1oMRAAFEv26ssE3izUAHR85dPb+A== --osd-uuid 8ed637ea-cc59-462d-be93-ce31ec32ba2c 2026-02-21T11:21:18.079 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:18.073+0000 7f037a446a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:18.080 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:18.077+0000 7f037a446a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:18.081 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:18.077+0000 7f037a446a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:18.082 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:18.077+0000 7f037a446a40 -1 bdev(0x555d449b0400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:18.082 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:18.077+0000 7f037a446a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:21:18.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:21:18.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:18.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:21:18.748 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:21:18.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:19.044 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:21:19.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:21:19.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:19.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:19.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:19.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:19.070 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:19.065+0000 7f20a444aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:19.072 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:19.069+0000 7f20a444aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:19.073 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:19.069+0000 7f20a444aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:19.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:21:19.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:19.541 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:19.537+0000 7f20a444aa40 -1 Falling back to public interface 2026-02-21T11:21:20.133 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:20.129+0000 7f20a444aa40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:21:20.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:20.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:20.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:20.474 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:20.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:20.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:21:20.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:21.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:21.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:21.691 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:21.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:21:21.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:21.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:21:21.907 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3701303367,v1:127.0.0.1:6835/3701303367] [v2:127.0.0.1:6836/3701303367,v1:127.0.0.1:6837/3701303367] exists,up 8ed637ea-cc59-462d-be93-ce31ec32ba2c 2026-02-21T11:21:21.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:21.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:21.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:21.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:21.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-21T11:21:21.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:21.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:21.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-21T11:21:21.914 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:21.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4c846a5b-5b39-483b-95b6-60ce846e920c 2026-02-21T11:21:21.915 INFO:tasks.workunit.client.0.trial166.stdout:add osd5 4c846a5b-5b39-483b-95b6-60ce846e920c 2026-02-21T11:21:21.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 4c846a5b-5b39-483b-95b6-60ce846e920c' 2026-02-21T11:21:21.915 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:21.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAxlZlpCvQKNxAAlaAJYtxe4oxQpfQ9k4w63Q== 2026-02-21T11:21:21.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAxlZlpCvQKNxAAlaAJYtxe4oxQpfQ9k4w63Q=="}' 2026-02-21T11:21:21.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4c846a5b-5b39-483b-95b6-60ce846e920c -i td/test-erasure-eio/5/new.json 2026-02-21T11:21:22.144 INFO:tasks.workunit.client.0.trial166.stdout:5 2026-02-21T11:21:22.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-21T11:21:22.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAxlZlpCvQKNxAAlaAJYtxe4oxQpfQ9k4w63Q== --osd-uuid 4c846a5b-5b39-483b-95b6-60ce846e920c 2026-02-21T11:21:22.182 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:22.177+0000 7f820a746a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:22.183 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:22.181+0000 7f820a746a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:22.184 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:22.181+0000 7f820a746a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:22.185 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:22.181+0000 7f820a746a40 -1 bdev(0x55a05f9ae400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:22.185 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:22.181+0000 7f820a746a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-21T11:21:22.838 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-21T11:21:22.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:22.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-21T11:21:22.840 INFO:tasks.workunit.client.0.trial166.stdout:adding osd5 key to auth repository 2026-02-21T11:21:22.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:23.139 INFO:tasks.workunit.client.0.trial166.stdout:start osd.5 2026-02-21T11:21:23.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-21T11:21:23.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:23.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:23.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:23.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:23.158 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:23.153+0000 7f46ff3f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:23.159 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:23.157+0000 7f46ff3f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:23.160 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:23.157+0000 7f46ff3f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:23.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-21T11:21:23.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:23.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:21:23.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:23.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:23.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:23.356 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:23.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:23.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:23.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:21:23.529 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:23.525+0000 7f46ff3f2a40 -1 Falling back to public interface 2026-02-21T11:21:23.573 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:24.078 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:24.073+0000 7f46ff3f2a40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:21:24.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:24.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:24.575 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:24.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:24.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:24.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:21:24.797 INFO:tasks.workunit.client.0.trial166.stdout:osd.5 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2453829399,v1:127.0.0.1:6843/2453829399] [v2:127.0.0.1:6844/2453829399,v1:127.0.0.1:6845/2453829399] exists,up 4c846a5b-5b39-483b-95b6-60ce846e920c 2026-02-21T11:21:24.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:24.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:24.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:24.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:24.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-21T11:21:24.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:24.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:24.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-21T11:21:24.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:24.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-21T11:21:24.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:24.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:24.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:24.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:24.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-21T11:21:24.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-21T11:21:24.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:24.806 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:24.807 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:24.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:24.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:24.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:24.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:24.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:24.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:24.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:24.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:24.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:24.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:24.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:24.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:24.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:24.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-21T11:21:24.809 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:24.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b7696c71-b17b-4670-b7df-0d95d42e2cfb 2026-02-21T11:21:24.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 b7696c71-b17b-4670-b7df-0d95d42e2cfb' 2026-02-21T11:21:24.810 INFO:tasks.workunit.client.0.trial166.stdout:add osd6 b7696c71-b17b-4670-b7df-0d95d42e2cfb 2026-02-21T11:21:24.811 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:24.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA0lZlpfEJsMBAA0RWdxpIF35AJdp2vWSc7kw== 2026-02-21T11:21:24.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA0lZlpfEJsMBAA0RWdxpIF35AJdp2vWSc7kw=="}' 2026-02-21T11:21:24.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b7696c71-b17b-4670-b7df-0d95d42e2cfb -i td/test-erasure-eio/6/new.json 2026-02-21T11:21:25.032 INFO:tasks.workunit.client.0.trial166.stdout:6 2026-02-21T11:21:25.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-21T11:21:25.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA0lZlpfEJsMBAA0RWdxpIF35AJdp2vWSc7kw== --osd-uuid b7696c71-b17b-4670-b7df-0d95d42e2cfb 2026-02-21T11:21:25.061 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:25.057+0000 7f8e49c72a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:25.062 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:25.057+0000 7f8e49c72a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:25.064 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:25.061+0000 7f8e49c72a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:25.064 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:25.061+0000 7f8e49c72a40 -1 bdev(0x5595b15a6400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:25.064 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:25.061+0000 7f8e49c72a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-21T11:21:25.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-21T11:21:25.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:25.727 INFO:tasks.workunit.client.0.trial166.stdout:adding osd6 key to auth repository 2026-02-21T11:21:25.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-21T11:21:25.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:26.028 INFO:tasks.workunit.client.0.trial166.stdout:start osd.6 2026-02-21T11:21:26.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-21T11:21:26.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:26.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:26.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:26.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:26.054 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:26.049+0000 7f77f9a75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:26.055 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:26.053+0000 7f77f9a75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:26.056 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:26.053+0000 7f77f9a75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:26.243 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-21T11:21:26.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:26.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:21:26.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:26.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:26.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:26.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:26.245 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:26.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:26.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:21:26.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:26.469 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:26.465+0000 7f77f9a75a40 -1 Falling back to public interface 2026-02-21T11:21:27.108 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:27.105+0000 7f77f9a75a40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:21:27.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:27.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:27.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:27.459 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:27.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:27.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:21:27.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:28.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:28.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:28.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:21:28.689 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:28.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:28.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:21:28.902 INFO:tasks.workunit.client.0.trial166.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2134284179,v1:127.0.0.1:6851/2134284179] [v2:127.0.0.1:6852/2134284179,v1:127.0.0.1:6853/2134284179] exists,up b7696c71-b17b-4670-b7df-0d95d42e2cfb 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:28.904 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:28.905 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:21:28.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:21:28.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:21:28.961 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:21:28.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:21:28.970 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:21:04.529+0000 7fdd2d2d0a40 0 load: jerasure load: lrc 2026-02-21T11:21:28.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:21:28.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-21T11:21:28.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:21:28.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-21T11:21:29.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:21:29.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:21:29.619 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:21:29.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:21:30.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:21:30.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:21:30.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:21:30.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:21:30.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:21:30.634 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:21:30.634 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:21:30.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:21:30.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:21:30.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:21:30.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:21:30.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:21:30.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:21:30.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:21:30.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:21:30.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:21:30.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:30.955 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:21:31.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803783 2026-02-21T11:21:31.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803783 2026-02-21T11:21:31.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783' 2026-02-21T11:21:31.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:31.037 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:21:31.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574855 2026-02-21T11:21:31.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574855 2026-02-21T11:21:31.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574855' 2026-02-21T11:21:31.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:31.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:21:31.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378630 2026-02-21T11:21:31.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378630 2026-02-21T11:21:31.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630' 2026-02-21T11:21:31.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:31.196 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:21:31.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149702 2026-02-21T11:21:31.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149702 2026-02-21T11:21:31.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702' 2026-02-21T11:21:31.278 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:31.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:21:31.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822661 2026-02-21T11:21:31.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822661 2026-02-21T11:21:31.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-154618822661' 2026-02-21T11:21:31.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:31.357 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:21:31.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=188978561028 2026-02-21T11:21:31.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 188978561028 2026-02-21T11:21:31.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-154618822661 5-188978561028' 2026-02-21T11:21:31.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:31.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:21:31.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=223338299396 2026-02-21T11:21:31.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 223338299396 2026-02-21T11:21:31.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-154618822661 5-188978561028 6-223338299396' 2026-02-21T11:21:31.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:31.513 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803783 2026-02-21T11:21:31.513 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:31.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:21:31.514 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803783 2026-02-21T11:21:31.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:31.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803783 2026-02-21T11:21:31.516 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803783 2026-02-21T11:21:31.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-21T11:21:31.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:21:31.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-21T11:21:31.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:21:32.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:21:32.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:21:32.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803783 2026-02-21T11:21:32.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:32.954 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574855 2026-02-21T11:21:32.954 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:32.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:21:32.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574855 2026-02-21T11:21:32.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:32.958 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574855 2026-02-21T11:21:32.958 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-21T11:21:32.958 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574855 2026-02-21T11:21:32.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:21:33.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-21T11:21:33.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:33.184 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378630 2026-02-21T11:21:33.184 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:33.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:21:33.186 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378630 2026-02-21T11:21:33.186 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:33.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378630 2026-02-21T11:21:33.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378630' 2026-02-21T11:21:33.188 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378630 2026-02-21T11:21:33.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:21:33.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378630 -lt 81604378630 2026-02-21T11:21:33.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:33.406 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149702 2026-02-21T11:21:33.406 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:33.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:21:33.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149702 2026-02-21T11:21:33.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:33.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149702 2026-02-21T11:21:33.410 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149702 2026-02-21T11:21:33.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-21T11:21:33.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:21:33.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-21T11:21:33.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:33.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822661 2026-02-21T11:21:33.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:33.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:21:33.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822661 2026-02-21T11:21:33.634 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:33.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822661 2026-02-21T11:21:33.636 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 154618822661 2026-02-21T11:21:33.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822661' 2026-02-21T11:21:33.636 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:21:33.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-21T11:21:33.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:33.847 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-188978561028 2026-02-21T11:21:33.847 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:33.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:21:33.849 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-188978561028 2026-02-21T11:21:33.849 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:33.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=188978561028 2026-02-21T11:21:33.851 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 188978561028 2026-02-21T11:21:33.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-21T11:21:33.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:21:34.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-21T11:21:34.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:34.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-223338299396 2026-02-21T11:21:34.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:34.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:21:34.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-223338299396 2026-02-21T11:21:34.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:34.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=223338299396 2026-02-21T11:21:34.074 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 223338299396 2026-02-21T11:21:34.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 223338299396' 2026-02-21T11:21:34.074 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:21:34.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 223338299397 -lt 223338299396 2026-02-21T11:21:34.293 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:21:34.294 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:21:34.294 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:21:34.569 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:21:34.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:21:34.570 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:21:34.570 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:21:34.570 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:21:34.570 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:21:34.570 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:21:34.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:21:34.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:21:34.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:21:34.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:21:34.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:21:35.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:21:35.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:21:35.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:21:35.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:21:35.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:21:35.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-21T11:21:35.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-21T11:21:35.396 INFO:tasks.workunit.client.0.trial166.stderr:osd.0 is already out. 2026-02-21T11:21:35.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-21T11:21:35.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:21:35.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:21:35.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:21:35.406 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:21:35.407 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:21:35.407 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:21:35.407 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:21:35.407 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:21:35.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:21:35.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:21:35.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:21:35.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:21:35.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:21:35.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:21:35.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:21:35.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:35.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:21:35.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803786 2026-02-21T11:21:35.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803786 2026-02-21T11:21:35.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786' 2026-02-21T11:21:35.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:35.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:21:35.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574858 2026-02-21T11:21:35.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574858 2026-02-21T11:21:35.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574858' 2026-02-21T11:21:35.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:35.888 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:21:35.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378633 2026-02-21T11:21:35.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378633 2026-02-21T11:21:35.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633' 2026-02-21T11:21:35.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:35.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:21:36.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149705 2026-02-21T11:21:36.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149705 2026-02-21T11:21:36.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705' 2026-02-21T11:21:36.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:36.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:21:36.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822663 2026-02-21T11:21:36.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822663 2026-02-21T11:21:36.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-154618822663' 2026-02-21T11:21:36.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:36.125 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:21:36.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=188978561031 2026-02-21T11:21:36.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 188978561031 2026-02-21T11:21:36.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-154618822663 5-188978561031' 2026-02-21T11:21:36.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:21:36.204 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:21:36.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=223338299399 2026-02-21T11:21:36.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 223338299399 2026-02-21T11:21:36.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-154618822663 5-188978561031 6-223338299399' 2026-02-21T11:21:36.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:36.281 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803786 2026-02-21T11:21:36.281 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:36.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:21:36.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803786 2026-02-21T11:21:36.283 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:36.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803786 2026-02-21T11:21:36.284 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803786 2026-02-21T11:21:36.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-21T11:21:36.285 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:21:36.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-21T11:21:36.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:36.501 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574858 2026-02-21T11:21:36.501 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:36.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:21:36.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574858 2026-02-21T11:21:36.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:36.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574858 2026-02-21T11:21:36.504 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574858 2026-02-21T11:21:36.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-21T11:21:36.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:21:36.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-21T11:21:36.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:36.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378633 2026-02-21T11:21:36.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:36.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:21:36.725 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378633 2026-02-21T11:21:36.725 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:36.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378633 2026-02-21T11:21:36.727 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378633 2026-02-21T11:21:36.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378633' 2026-02-21T11:21:36.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:21:36.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378633 -lt 81604378633 2026-02-21T11:21:36.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:36.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149705 2026-02-21T11:21:36.953 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:36.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:21:36.955 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149705 2026-02-21T11:21:36.955 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:36.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149705 2026-02-21T11:21:36.956 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149705 2026-02-21T11:21:36.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-21T11:21:36.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:21:37.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-21T11:21:37.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:37.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822663 2026-02-21T11:21:37.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:37.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:21:37.190 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822663 2026-02-21T11:21:37.190 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:37.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822663 2026-02-21T11:21:37.191 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 154618822663 2026-02-21T11:21:37.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822663' 2026-02-21T11:21:37.191 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:21:37.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822661 -lt 154618822663 2026-02-21T11:21:37.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:21:38.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:21:38.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:21:38.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822664 -lt 154618822663 2026-02-21T11:21:38.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:38.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-188978561031 2026-02-21T11:21:38.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:38.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:21:38.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-188978561031 2026-02-21T11:21:38.628 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:38.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=188978561031 2026-02-21T11:21:38.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 188978561031' 2026-02-21T11:21:38.629 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 188978561031 2026-02-21T11:21:38.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:21:38.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 188978561031 -lt 188978561031 2026-02-21T11:21:38.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:21:38.849 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-223338299399 2026-02-21T11:21:38.849 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:21:38.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:21:38.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-223338299399 2026-02-21T11:21:38.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:21:38.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=223338299399 2026-02-21T11:21:38.852 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 223338299399 2026-02-21T11:21:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 223338299399' 2026-02-21T11:21:38.852 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:21:39.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 223338299399 -lt 223338299399 2026-02-21T11:21:39.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:21:39.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:21:39.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:21:39.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:21:39.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:21:39.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:21:39.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:21:39.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:21:39.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:21:39.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:21:39.356 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:21:39.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:21:39.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:21:39.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:21:39.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:21:39.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:21:39.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:21:39.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:21:39.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-21T11:21:39.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:21:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:21:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-21T11:21:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:21:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:21:39.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-21T11:21:39.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:21:39.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:21:39.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-21T11:21:39.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:21:39.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:21:39.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-21T11:21:39.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:21:39.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:21:39.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-21T11:21:39.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:21:39.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:21:39.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-21T11:21:39.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:21:39.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:21:39.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:21:39.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:21:39.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:21:40.183 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:21:40.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:21:40.466 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:21:40.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:21:40.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:21:40.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:21:40.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:21:40.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:21:40.792 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:21:40.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:21:40.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:21:40.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:21:40.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:40.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:21:40.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:21:40.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:40.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:21:40.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:21:40.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:21:40.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:21:40.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:40.816 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:40.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:21:40.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:21:40.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:21:40.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:21:40.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:21:40.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:21:40.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:21:40.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:21:40.820 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:21:40.821 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:21:40.821 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:21:40.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:21:40.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:21:40.822 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:21:40.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:40.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:21:40.824 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:21:40.824 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:21:40.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:21:40.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:21:40.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:21:40.828 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:21:40.828 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:40.828 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:40.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:21:40.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:21:40.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:21:40.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:21:40.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:21:40.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:40.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:40.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:21:40.832 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:21:40.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:21:40.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:21:40.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:21:40.854 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:21:40.854 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:40.854 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:40.855 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:40.855 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:40.855 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:40.855 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:40.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:21:40.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:21:40.884 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:21:40.884 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:21:40.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:21:40.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:21:40.886 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:21:40.886 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:21:40.886 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:21:40.886 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:21:40.886 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:40.886 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:40.887 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:40.887 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:21:40.887 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:21:40.887 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:21:40.944 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:40.945 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:40.945 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:40.945 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:21:40.945 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:21:40.946 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:21:41.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:21:41.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:21:41.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:21:41.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:21:41.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:21:41.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:21:41.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:21:41.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:21:41.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:41.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:41.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:41.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:41.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:41.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:41.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:21:41.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:21:41.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:21:41.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:21:41.243 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:21:41.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:21:42.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:21:42.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:21:42.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:21:42.253 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:21:42.253 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:42.253 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:42.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:21:42.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:21:42.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:21:42.304 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:21:42.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:21:42.311 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:21:40.869+0000 7f547f271ec0 0 load: jerasure load: lrc 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:21:42.312 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-21T11:21:42.313 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-21T11:21:42.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:42.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:42.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:42.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:42.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:42.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:42.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:42.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:21:42.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:42.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fcaf5fd1-6390-46ce-a306-59bf63f1d1ab 2026-02-21T11:21:42.318 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 fcaf5fd1-6390-46ce-a306-59bf63f1d1ab 2026-02-21T11:21:42.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fcaf5fd1-6390-46ce-a306-59bf63f1d1ab' 2026-02-21T11:21:42.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:42.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBGlZlpXP2IExAA67W6YcB2KRDZWkkxKi2Aww== 2026-02-21T11:21:42.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBGlZlpXP2IExAA67W6YcB2KRDZWkkxKi2Aww=="}' 2026-02-21T11:21:42.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fcaf5fd1-6390-46ce-a306-59bf63f1d1ab -i td/test-erasure-eio/0/new.json 2026-02-21T11:21:42.446 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:42.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:21:42.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBGlZlpXP2IExAA67W6YcB2KRDZWkkxKi2Aww== --osd-uuid fcaf5fd1-6390-46ce-a306-59bf63f1d1ab 2026-02-21T11:21:42.473 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:42.469+0000 7fd61e177a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:42.476 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:42.473+0000 7fd61e177a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:42.477 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:42.473+0000 7fd61e177a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:42.477 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:42.473+0000 7fd61e177a40 -1 bdev(0x5635704f8400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:42.477 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:42.473+0000 7fd61e177a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:21:43.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:21:43.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:43.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:21:43.159 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:21:43.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:43.315 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:21:43.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:21:43.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:43.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:43.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:43.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:43.337 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:43.333+0000 7f390249ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:43.339 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:43.333+0000 7f390249ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:43.340 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:43.337+0000 7f390249ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:43.431 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:43.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:43.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:43.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:21:43.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:43.713 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:43.709+0000 7f390249ba40 -1 Falling back to public interface 2026-02-21T11:21:44.337 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:44.333+0000 7f390249ba40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:21:44.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:44.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:44.643 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:44.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:44.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:44.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:21:44.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:45.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:45.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:45.858 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:45.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:21:45.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:45.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:21:46.080 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2919914409,v1:127.0.0.1:6803/2919914409] [v2:127.0.0.1:6804/2919914409,v1:127.0.0.1:6805/2919914409] exists,up fcaf5fd1-6390-46ce-a306-59bf63f1d1ab 2026-02-21T11:21:46.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:21:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:46.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:21:46.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:46.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:21:46.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:46.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:46.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:46.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:46.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:21:46.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:21:46.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:46.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:46.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:46.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:46.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:46.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:46.090 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:46.090 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:46.090 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:46.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:46.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:46.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:46.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:46.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:46.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:46.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:46.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:21:46.093 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:46.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ae528d25-9935-4e89-8ddf-1e988e5ccc45 2026-02-21T11:21:46.094 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 ae528d25-9935-4e89-8ddf-1e988e5ccc45 2026-02-21T11:21:46.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ae528d25-9935-4e89-8ddf-1e988e5ccc45' 2026-02-21T11:21:46.095 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:46.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBKlZlpH/fDBRAAXdLr6Y0CNqeJiLcQ42Hbvw== 2026-02-21T11:21:46.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBKlZlpH/fDBRAAXdLr6Y0CNqeJiLcQ42Hbvw=="}' 2026-02-21T11:21:46.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ae528d25-9935-4e89-8ddf-1e988e5ccc45 -i td/test-erasure-eio/1/new.json 2026-02-21T11:21:46.309 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:46.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:21:46.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBKlZlpH/fDBRAAXdLr6Y0CNqeJiLcQ42Hbvw== --osd-uuid ae528d25-9935-4e89-8ddf-1e988e5ccc45 2026-02-21T11:21:46.339 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:46.333+0000 7f597903fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:46.340 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:46.337+0000 7f597903fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:46.341 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:46.337+0000 7f597903fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:46.342 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:46.337+0000 7f597903fa40 -1 bdev(0x561d11de8400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:46.342 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:46.337+0000 7f597903fa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:21:46.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:21:46.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:46.962 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:21:46.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:21:46.963 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:47.258 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:21:47.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:21:47.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:47.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:47.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:47.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:47.285 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:47.281+0000 7f869190ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:47.287 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:47.285+0000 7f869190ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:47.288 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:47.285+0000 7f869190ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:47.475 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:47.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:47.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:47.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:21:47.649 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:47.645+0000 7f869190ea40 -1 Falling back to public interface 2026-02-21T11:21:47.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:48.188 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:48.185+0000 7f869190ea40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:21:48.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:48.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:48.692 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:48.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:48.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:48.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:21:48.906 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3853421350,v1:127.0.0.1:6811/3853421350] [v2:127.0.0.1:6812/3853421350,v1:127.0.0.1:6813/3853421350] exists,up ae528d25-9935-4e89-8ddf-1e988e5ccc45 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:21:48.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:48.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:48.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5ac3ad9b-6622-45ef-881b-aa9c1b35ff50 2026-02-21T11:21:48.914 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 5ac3ad9b-6622-45ef-881b-aa9c1b35ff50 2026-02-21T11:21:48.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5ac3ad9b-6622-45ef-881b-aa9c1b35ff50' 2026-02-21T11:21:48.914 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:48.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBMlZlpaEnsNhAA+XRoTsg+5S9S6VktOSpdKQ== 2026-02-21T11:21:48.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBMlZlpaEnsNhAA+XRoTsg+5S9S6VktOSpdKQ=="}' 2026-02-21T11:21:48.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5ac3ad9b-6622-45ef-881b-aa9c1b35ff50 -i td/test-erasure-eio/2/new.json 2026-02-21T11:21:49.134 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:21:49.143 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:21:49.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBMlZlpaEnsNhAA+XRoTsg+5S9S6VktOSpdKQ== --osd-uuid 5ac3ad9b-6622-45ef-881b-aa9c1b35ff50 2026-02-21T11:21:49.162 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:49.157+0000 7fe7ba1b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:49.164 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:49.161+0000 7fe7ba1b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:49.165 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:49.161+0000 7fe7ba1b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:49.165 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:49.161+0000 7fe7ba1b2a40 -1 bdev(0x55800dad0400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:49.165 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:49.161+0000 7fe7ba1b2a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:21:49.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:21:49.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:49.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:21:49.803 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:21:49.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:50.090 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:21:50.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:21:50.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:50.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:50.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:50.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:50.118 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:50.113+0000 7f075a2eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:50.120 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:50.117+0000 7f075a2eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:50.121 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:50.117+0000 7f075a2eda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:50.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:50.312 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:50.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:50.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:21:50.497 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:50.493+0000 7f075a2eda40 -1 Falling back to public interface 2026-02-21T11:21:50.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:51.109 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:51.105+0000 7f075a2eda40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:21:51.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:51.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:51.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:51.524 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:51.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:51.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:21:51.741 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1987393986,v1:127.0.0.1:6819/1987393986] [v2:127.0.0.1:6820/1987393986,v1:127.0.0.1:6821/1987393986] exists,up 5ac3ad9b-6622-45ef-881b-aa9c1b35ff50 2026-02-21T11:21:51.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:51.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:51.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:51.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:51.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:21:51.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:51.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:51.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:21:51.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:51.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:21:51.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:51.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:51.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:51.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:51.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:21:51.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:21:51.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:51.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:51.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:51.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:51.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:51.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:51.752 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:51.752 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:51.752 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:51.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:51.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:51.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:51.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:51.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:51.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:51.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:51.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:21:51.755 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:51.755 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c4d725c8-7967-412f-8d4f-6f42cb8ee748 2026-02-21T11:21:51.755 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 c4d725c8-7967-412f-8d4f-6f42cb8ee748 2026-02-21T11:21:51.756 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c4d725c8-7967-412f-8d4f-6f42cb8ee748' 2026-02-21T11:21:51.756 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:51.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBPlZlp9fUcLRAATMsrRjlBDaYjBNF/YM9ZTg== 2026-02-21T11:21:51.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBPlZlp9fUcLRAATMsrRjlBDaYjBNF/YM9ZTg=="}' 2026-02-21T11:21:51.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c4d725c8-7967-412f-8d4f-6f42cb8ee748 -i td/test-erasure-eio/3/new.json 2026-02-21T11:21:51.971 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:21:51.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:21:51.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBPlZlp9fUcLRAATMsrRjlBDaYjBNF/YM9ZTg== --osd-uuid c4d725c8-7967-412f-8d4f-6f42cb8ee748 2026-02-21T11:21:51.999 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:51.993+0000 7f5836eb1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:52.000 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:51.997+0000 7f5836eb1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:52.001 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:51.997+0000 7f5836eb1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:52.002 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:51.997+0000 7f5836eb1a40 -1 bdev(0x55f4815d4400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:52.002 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:51.997+0000 7f5836eb1a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:21:52.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:21:52.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:52.667 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:21:52.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:21:52.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:52.973 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:21:52.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:21:52.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:52.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:52.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:52.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:52.990 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:52.984+0000 7f705d8d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:52.992 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:52.988+0000 7f705d8d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:52.993 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:52.988+0000 7f705d8d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:53.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:21:53.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:53.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:21:53.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:53.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:53.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:53.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:53.190 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:53.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:53.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:21:53.361 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:53.356+0000 7f705d8d3a40 -1 Falling back to public interface 2026-02-21T11:21:53.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:53.946 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:53.940+0000 7f705d8d3a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:21:54.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:54.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:54.406 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:54.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:54.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:54.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:21:54.612 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2549891833,v1:127.0.0.1:6827/2549891833] [v2:127.0.0.1:6828/2549891833,v1:127.0.0.1:6829/2549891833] exists,up c4d725c8-7967-412f-8d4f-6f42cb8ee748 2026-02-21T11:21:54.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:54.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:54.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:54.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:54.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:21:54.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:54.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:21:54.619 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:54.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8a7a1abe-9ba1-405b-a0cc-9893223e8f10 2026-02-21T11:21:54.620 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 8a7a1abe-9ba1-405b-a0cc-9893223e8f10 2026-02-21T11:21:54.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 8a7a1abe-9ba1-405b-a0cc-9893223e8f10' 2026-02-21T11:21:54.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:54.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBSlZlpWw13JRAApRh0dRa87Kpx/4TVKOyrYw== 2026-02-21T11:21:54.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBSlZlpWw13JRAApRh0dRa87Kpx/4TVKOyrYw=="}' 2026-02-21T11:21:54.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8a7a1abe-9ba1-405b-a0cc-9893223e8f10 -i td/test-erasure-eio/4/new.json 2026-02-21T11:21:54.855 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:21:54.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:21:54.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBSlZlpWw13JRAApRh0dRa87Kpx/4TVKOyrYw== --osd-uuid 8a7a1abe-9ba1-405b-a0cc-9893223e8f10 2026-02-21T11:21:54.883 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:54.880+0000 7f8b0ef52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:54.884 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:54.880+0000 7f8b0ef52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:54.885 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:54.880+0000 7f8b0ef52a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:54.886 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:54.880+0000 7f8b0ef52a40 -1 bdev(0x555764298400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:54.886 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:54.880+0000 7f8b0ef52a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:21:55.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:21:55.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:55.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:21:55.571 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:21:55.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:55.861 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:21:55.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:21:55.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:55.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:55.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:55.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:55.889 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:55.884+0000 7f83c303fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:55.891 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:55.884+0000 7f83c303fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:55.892 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:55.888+0000 7f83c303fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:56.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:21:56.264 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:56.260+0000 7f83c303fa40 -1 Falling back to public interface 2026-02-21T11:21:56.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:56.870 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:56.864+0000 7f83c303fa40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:21:57.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:21:57.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:57.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:21:57.296 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:21:57.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:57.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:21:57.514 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3228150732,v1:127.0.0.1:6835/3228150732] [v2:127.0.0.1:6836/3228150732,v1:127.0.0.1:6837/3228150732] exists,up 8a7a1abe-9ba1-405b-a0cc-9893223e8f10 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:21:57.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:21:57.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:21:57.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-21T11:21:57.518 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:21:57.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=705e4e82-f009-459f-a63d-a599707299c5 2026-02-21T11:21:57.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 705e4e82-f009-459f-a63d-a599707299c5' 2026-02-21T11:21:57.519 INFO:tasks.workunit.client.0.trial166.stdout:add osd5 705e4e82-f009-459f-a63d-a599707299c5 2026-02-21T11:21:57.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:21:57.534 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBVlZlpHTKuHxAAtjp4xAEIhkG+HBROlet+sA== 2026-02-21T11:21:57.534 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBVlZlpHTKuHxAAtjp4xAEIhkG+HBROlet+sA=="}' 2026-02-21T11:21:57.534 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 705e4e82-f009-459f-a63d-a599707299c5 -i td/test-erasure-eio/5/new.json 2026-02-21T11:21:57.759 INFO:tasks.workunit.client.0.trial166.stdout:5 2026-02-21T11:21:57.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-21T11:21:57.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBVlZlpHTKuHxAAtjp4xAEIhkG+HBROlet+sA== --osd-uuid 705e4e82-f009-459f-a63d-a599707299c5 2026-02-21T11:21:57.796 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:57.792+0000 7f396f10ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:57.798 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:57.792+0000 7f396f10ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:57.799 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:57.796+0000 7f396f10ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:57.800 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:57.796+0000 7f396f10ca40 -1 bdev(0x55d757128400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-21T11:21:57.800 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:57.796+0000 7f396f10ca40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-21T11:21:58.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-21T11:21:58.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:21:58.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-21T11:21:58.467 INFO:tasks.workunit.client.0.trial166.stdout:adding osd5 key to auth repository 2026-02-21T11:21:58.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:21:58.764 INFO:tasks.workunit.client.0.trial166.stdout:start osd.5 2026-02-21T11:21:58.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-21T11:21:58.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:21:58.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:21:58.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:21:58.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:21:58.782 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:58.776+0000 7f0a5ec5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:58.783 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:58.780+0000 7f0a5ec5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:58.784 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:58.780+0000 7f0a5ec5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:21:58.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:21:58.985 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:21:58.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:21:58.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:21:59.181 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:59.176+0000 7f0a5ec5ba40 -1 Falling back to public interface 2026-02-21T11:21:59.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:21:59.804 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:21:59.800+0000 7f0a5ec5ba40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:22:00.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:00.204 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:00.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:00.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:00.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:00.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:22:00.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:01.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:01.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:01.417 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:22:01.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:22:01.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:01.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:22:01.634 INFO:tasks.workunit.client.0.trial166.stdout:osd.5 up in weight 1 up_from 42 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/4214875022,v1:127.0.0.1:6843/4214875022] [v2:127.0.0.1:6844/4214875022,v1:127.0.0.1:6845/4214875022] exists,up 705e4e82-f009-459f-a63d-a599707299c5 2026-02-21T11:22:01.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:22:01.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:01.639 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:01.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:01.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:01.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:22:01.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:22:01.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-21T11:22:01.641 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:22:01.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a7198f98-0ace-48bd-9e3f-d37cbc118cd0 2026-02-21T11:22:01.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 a7198f98-0ace-48bd-9e3f-d37cbc118cd0' 2026-02-21T11:22:01.642 INFO:tasks.workunit.client.0.trial166.stdout:add osd6 a7198f98-0ace-48bd-9e3f-d37cbc118cd0 2026-02-21T11:22:01.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:22:01.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZlZlptkuqJhAA9df7j4gIDIPIN3IPGTN/+Q== 2026-02-21T11:22:01.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZlZlptkuqJhAA9df7j4gIDIPIN3IPGTN/+Q=="}' 2026-02-21T11:22:01.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a7198f98-0ace-48bd-9e3f-d37cbc118cd0 -i td/test-erasure-eio/6/new.json 2026-02-21T11:22:01.878 INFO:tasks.workunit.client.0.trial166.stdout:6 2026-02-21T11:22:01.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-21T11:22:01.890 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZlZlptkuqJhAA9df7j4gIDIPIN3IPGTN/+Q== --osd-uuid a7198f98-0ace-48bd-9e3f-d37cbc118cd0 2026-02-21T11:22:01.915 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:01.912+0000 7f3d36385a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:01.917 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:01.912+0000 7f3d36385a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:01.918 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:01.912+0000 7f3d36385a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:01.919 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:01.912+0000 7f3d36385a40 -1 bdev(0x55b017290400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-21T11:22:01.919 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:01.916+0000 7f3d36385a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-21T11:22:02.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-21T11:22:02.578 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:22:02.578 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-21T11:22:02.579 INFO:tasks.workunit.client.0.trial166.stdout:adding osd6 key to auth repository 2026-02-21T11:22:02.579 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:22:02.868 INFO:tasks.workunit.client.0.trial166.stdout:start osd.6 2026-02-21T11:22:02.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-21T11:22:02.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:02.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:22:02.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:22:02.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:22:02.895 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:02.888+0000 7f7bda403a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:02.896 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:02.892+0000 7f7bda403a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:02.897 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:02.892+0000 7f7bda403a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:03.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-21T11:22:03.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:03.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:22:03.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:03.373 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:03.368+0000 7f7bda403a40 -1 Falling back to public interface 2026-02-21T11:22:03.962 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:03.956+0000 7f7bda403a40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:22:04.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:04.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:04.306 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:04.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:04.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:04.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:22:04.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:05.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:05.547 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:22:05.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:05.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:22:05.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:05.548 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:22:05.763 INFO:tasks.workunit.client.0.trial166.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/792624805,v1:127.0.0.1:6851/792624805] [v2:127.0.0.1:6852/792624805,v1:127.0.0.1:6853/792624805] exists,up a7198f98-0ace-48bd-9e3f-d37cbc118cd0 2026-02-21T11:22:05.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:22:05.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:22:05.820 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:22:05.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:22:05.830 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:21:43.713+0000 7f390249ba40 0 load: jerasure load: lrc 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:22:05.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-21T11:22:06.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:22:06.108 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:22:06.478 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:22:06.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:22:07.490 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:22:07.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:22:07.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:22:07.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:22:07.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:22:07.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:22:07.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:22:07.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:22:07.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:22:07.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:22:07.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:22:07.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:22:07.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:22:07.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:22:07.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:22:07.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:22:07.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:22:07.809 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:07.809 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:22:07.809 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:22:07.809 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:22:07.809 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:22:07.810 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:22:07.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:22:07.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:07.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:22:07.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803783 2026-02-21T11:22:07.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803783 2026-02-21T11:22:07.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783' 2026-02-21T11:22:07.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:07.888 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:22:07.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607558 2026-02-21T11:22:07.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607558 2026-02-21T11:22:07.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607558' 2026-02-21T11:22:07.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:07.970 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:22:08.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444038 2026-02-21T11:22:08.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444038 2026-02-21T11:22:08.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-73014444038' 2026-02-21T11:22:08.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:08.047 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:22:08.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182406 2026-02-21T11:22:08.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182406 2026-02-21T11:22:08.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-73014444038 3-107374182406' 2026-02-21T11:22:08.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:08.126 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:22:08.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920774 2026-02-21T11:22:08.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920774 2026-02-21T11:22:08.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-73014444038 3-107374182406 4-141733920774' 2026-02-21T11:22:08.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:08.204 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:22:08.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626436 2026-02-21T11:22:08.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626436 2026-02-21T11:22:08.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-73014444038 3-107374182406 4-141733920774 5-180388626436' 2026-02-21T11:22:08.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:08.284 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:22:08.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364803 2026-02-21T11:22:08.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364803 2026-02-21T11:22:08.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-73014444038 3-107374182406 4-141733920774 5-180388626436 6-214748364803' 2026-02-21T11:22:08.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:08.366 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803783 2026-02-21T11:22:08.366 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:08.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:22:08.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803783 2026-02-21T11:22:08.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:08.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803783 2026-02-21T11:22:08.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-21T11:22:08.369 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803783 2026-02-21T11:22:08.370 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:22:08.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-21T11:22:08.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:22:09.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:22:09.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:22:09.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-21T11:22:09.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:09.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607558 2026-02-21T11:22:09.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:09.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:22:09.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607558 2026-02-21T11:22:09.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:09.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607558 2026-02-21T11:22:09.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-21T11:22:09.810 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607558 2026-02-21T11:22:09.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:22:10.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-21T11:22:10.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:10.027 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444038 2026-02-21T11:22:10.027 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:10.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:22:10.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444038 2026-02-21T11:22:10.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:10.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444038 2026-02-21T11:22:10.031 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444038 2026-02-21T11:22:10.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444038' 2026-02-21T11:22:10.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:22:10.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444038 -lt 73014444038 2026-02-21T11:22:10.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:10.247 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182406 2026-02-21T11:22:10.247 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:10.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:22:10.249 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182406 2026-02-21T11:22:10.249 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:10.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182406 2026-02-21T11:22:10.251 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182406 2026-02-21T11:22:10.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-21T11:22:10.251 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:22:10.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-21T11:22:10.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:10.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920774 2026-02-21T11:22:10.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:10.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:22:10.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920774 2026-02-21T11:22:10.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:10.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920774 2026-02-21T11:22:10.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920774' 2026-02-21T11:22:10.468 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920774 2026-02-21T11:22:10.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:22:10.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-21T11:22:10.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:10.693 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-180388626436 2026-02-21T11:22:10.693 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:10.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:22:10.695 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-180388626436 2026-02-21T11:22:10.695 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:10.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626436 2026-02-21T11:22:10.696 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 180388626436 2026-02-21T11:22:10.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 180388626436' 2026-02-21T11:22:10.697 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:22:10.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626436 -lt 180388626436 2026-02-21T11:22:10.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:10.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-214748364803 2026-02-21T11:22:10.914 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:10.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:22:10.915 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-214748364803 2026-02-21T11:22:10.916 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:10.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364803 2026-02-21T11:22:10.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-21T11:22:10.917 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 214748364803 2026-02-21T11:22:10.917 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:22:11.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-21T11:22:11.138 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:22:11.138 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:22:11.139 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:22:11.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:22:11.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:22:11.631 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:22:11.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:22:11.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:22:11.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:11.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:22:11.935 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:11.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-21T11:22:11.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:22:11.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:22:11.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:22:11.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:11.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-21T11:22:11.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:22:11.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:22:11.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:22:11.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-21T11:22:11.976 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:22:11.977 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:22:11.977 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:22:11.977 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:22:12.187 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:22:12.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:22:12.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:22:12.189 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:22:12.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:22:12.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:22:12.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:22:12.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:22:12.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:22:12.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:22:12.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:22:12.192 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:22:12.192 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:22:12.192 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:22:12.193 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:12.193 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:12.193 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:12.193 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:22:12.193 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:22:12.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:22:12.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:12.254 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:12.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:22:12.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:22:12.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-21T11:22:12.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-21T11:22:12.574 INFO:tasks.workunit.client.0.trial166.stderr:osd.0 is already out. 2026-02-21T11:22:12.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-21T11:22:12.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:22:12.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:22:12.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:22:12.586 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:22:12.586 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:22:12.586 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:22:12.586 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:22:12.586 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:22:12.586 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:22:12.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:22:12.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:22:12.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:22:12.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:22:12.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:22:12.691 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:22:12.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:12.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:22:12.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803786 2026-02-21T11:22:12.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803786 2026-02-21T11:22:12.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786' 2026-02-21T11:22:12.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:12.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:22:13.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607561 2026-02-21T11:22:13.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607561 2026-02-21T11:22:13.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561' 2026-02-21T11:22:13.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:13.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:22:13.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444041 2026-02-21T11:22:13.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444041 2026-02-21T11:22:13.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-73014444041' 2026-02-21T11:22:13.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:13.146 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:22:13.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182409 2026-02-21T11:22:13.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182409 2026-02-21T11:22:13.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-73014444041 3-107374182409' 2026-02-21T11:22:13.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:13.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:22:13.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920777 2026-02-21T11:22:13.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920777 2026-02-21T11:22:13.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-73014444041 3-107374182409 4-141733920777' 2026-02-21T11:22:13.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:13.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:22:13.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626439 2026-02-21T11:22:13.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626439 2026-02-21T11:22:13.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-73014444041 3-107374182409 4-141733920777 5-180388626439' 2026-02-21T11:22:13.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:13.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:22:13.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364806 2026-02-21T11:22:13.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364806 2026-02-21T11:22:13.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-73014444041 3-107374182409 4-141733920777 5-180388626439 6-214748364806' 2026-02-21T11:22:13.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:13.466 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803786 2026-02-21T11:22:13.466 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:13.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:22:13.467 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803786 2026-02-21T11:22:13.468 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:13.469 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803786 2026-02-21T11:22:13.469 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803786 2026-02-21T11:22:13.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-21T11:22:13.470 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:22:13.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-21T11:22:13.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:13.687 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607561 2026-02-21T11:22:13.687 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:13.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:22:13.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607561 2026-02-21T11:22:13.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:13.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607561 2026-02-21T11:22:13.691 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607561 2026-02-21T11:22:13.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-21T11:22:13.691 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:22:13.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-21T11:22:13.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:13.908 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444041 2026-02-21T11:22:13.908 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:13.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:22:13.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444041 2026-02-21T11:22:13.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:13.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444041 2026-02-21T11:22:13.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444041' 2026-02-21T11:22:13.911 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444041 2026-02-21T11:22:13.911 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:22:14.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444041 -lt 73014444041 2026-02-21T11:22:14.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:14.132 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182409 2026-02-21T11:22:14.132 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:14.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:22:14.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182409 2026-02-21T11:22:14.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:14.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182409 2026-02-21T11:22:14.135 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182409 2026-02-21T11:22:14.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182409' 2026-02-21T11:22:14.136 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:22:14.350 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182409 -lt 107374182409 2026-02-21T11:22:14.350 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:14.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920777 2026-02-21T11:22:14.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:14.352 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:22:14.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920777 2026-02-21T11:22:14.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:14.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920777 2026-02-21T11:22:14.354 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920777 2026-02-21T11:22:14.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920777' 2026-02-21T11:22:14.354 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:22:14.563 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920777 -lt 141733920777 2026-02-21T11:22:14.563 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:14.564 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-180388626439 2026-02-21T11:22:14.564 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:14.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:22:14.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-180388626439 2026-02-21T11:22:14.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:14.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626439 2026-02-21T11:22:14.567 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 180388626439 2026-02-21T11:22:14.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 180388626439' 2026-02-21T11:22:14.567 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:22:14.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626437 -lt 180388626439 2026-02-21T11:22:14.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:22:15.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:22:15.785 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:22:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626439 -lt 180388626439 2026-02-21T11:22:16.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:16.000 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-214748364806 2026-02-21T11:22:16.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:16.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:22:16.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-214748364806 2026-02-21T11:22:16.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:16.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364806 2026-02-21T11:22:16.004 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 214748364806 2026-02-21T11:22:16.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 214748364806' 2026-02-21T11:22:16.004 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:22:16.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364807 -lt 214748364806 2026-02-21T11:22:16.227 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:22:16.227 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:22:16.227 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:22:16.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:22:16.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:22:16.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:22:16.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:22:16.732 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:22:16.732 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:22:17.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-21T11:22:17.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:22:17.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:22:17.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-21T11:22:17.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:22:17.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:22:17.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-21T11:22:17.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:22:17.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:22:17.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-21T11:22:17.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:22:17.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:22:17.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-21T11:22:17.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:22:17.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:22:17.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:22:17.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:22:17.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:22:17.327 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:22:17.338 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:22:17.615 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:22:17.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:22:17.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:22:17.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:22:17.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:22:17.629 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:22:17.629 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:22:17.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:22:17.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:22:17.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:22:17.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:22:17.740 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:22:17.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:22:17.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:22:17.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:22:17.742 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:22:17.743 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:22:17.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:22:17.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:22:17.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:22:17.744 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:22:17.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:22:17.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:22:17.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:22:17.768 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:22:17.768 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:17.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:17.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:22:17.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:22:17.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:22:17.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:22:17.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:22:17.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:22:17.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:22:17.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:22:17.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:22:17.774 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:22:17.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:22:17.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:22:17.776 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:22:17.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:22:17.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:22:17.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:22:17.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:22:17.779 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:22:17.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:22:17.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:22:17.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:22:17.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:17.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:17.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:22:17.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:22:17.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:22:17.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:22:17.784 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:22:17.784 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:17.784 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:17.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:22:17.786 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:22:17.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:22:17.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:22:17.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:22:17.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:22:17.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:22:17.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:17.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:17.809 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:17.809 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:17.809 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:17.809 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:17.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:22:17.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:22:17.840 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:17.841 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:17.842 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:17.842 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:22:17.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:22:17.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:22:17.897 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:22:17.898 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:22:17.899 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:17.899 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:17.899 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:17.899 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:22:17.899 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:22:17.899 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:22:17.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:22:18.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:22:18.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:18.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:18.063 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:18.063 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:18.064 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:18.064 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:18.064 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:22:18.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:22:18.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:22:18.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:22:18.190 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:22:18.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:22:19.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:22:19.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:22:19.251 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:22:19.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:22:19.258 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:22:17.824+0000 7f81dde5cec0 0 load: jerasure load: lrc 2026-02-21T11:22:19.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:22:19.259 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-21T11:22:19.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:22:19.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:19.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:22:19.263 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:22:19.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ce51c814-ecaa-402a-85e0-3d592c61f025 2026-02-21T11:22:19.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ce51c814-ecaa-402a-85e0-3d592c61f025' 2026-02-21T11:22:19.264 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 ce51c814-ecaa-402a-85e0-3d592c61f025 2026-02-21T11:22:19.264 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:22:19.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBrlZlp2VJnEBAATugLUCSk1KZI2Hir590/JQ== 2026-02-21T11:22:19.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBrlZlp2VJnEBAATugLUCSk1KZI2Hir590/JQ=="}' 2026-02-21T11:22:19.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ce51c814-ecaa-402a-85e0-3d592c61f025 -i td/test-erasure-eio/0/new.json 2026-02-21T11:22:19.390 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:19.397 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:22:19.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBrlZlp2VJnEBAATugLUCSk1KZI2Hir590/JQ== --osd-uuid ce51c814-ecaa-402a-85e0-3d592c61f025 2026-02-21T11:22:19.417 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:19.412+0000 7f1281e26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:19.418 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:19.412+0000 7f1281e26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:19.420 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:19.416+0000 7f1281e26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:19.420 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:19.416+0000 7f1281e26a40 -1 bdev(0x5614acedc400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:22:19.421 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:19.416+0000 7f1281e26a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:22:20.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:22:20.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:22:20.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:22:20.363 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:22:20.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:22:20.476 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:22:20.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:22:20.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:20.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:22:20.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:22:20.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:22:20.494 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:20.488+0000 7fbed2292a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:20.495 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:20.492+0000 7fbed2292a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:20.496 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:20.492+0000 7fbed2292a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:20.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:22:20.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:20.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:22:20.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:20.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:20.694 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:20.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:20.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:20.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:20.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:22:20.908 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:20.904+0000 7fbed2292a40 -1 Falling back to public interface 2026-02-21T11:22:20.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:21.532 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:21.528+0000 7fbed2292a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:22:21.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:21.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:21.918 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:21.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:21.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:21.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:22:22.129 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1848104459,v1:127.0.0.1:6803/1848104459] [v2:127.0.0.1:6804/1848104459,v1:127.0.0.1:6805/1848104459] exists,up ce51c814-ecaa-402a-85e0-3d592c61f025 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:22:22.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:22.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:22:22.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:22:22.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:22:22.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7422c443-245b-4aec-b886-978573d5b07d 2026-02-21T11:22:22.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7422c443-245b-4aec-b886-978573d5b07d' 2026-02-21T11:22:22.134 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 7422c443-245b-4aec-b886-978573d5b07d 2026-02-21T11:22:22.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:22:22.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBulZlpJqSsCBAA6121BXR0zcDiZ0cug7JPwQ== 2026-02-21T11:22:22.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBulZlpJqSsCBAA6121BXR0zcDiZ0cug7JPwQ=="}' 2026-02-21T11:22:22.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7422c443-245b-4aec-b886-978573d5b07d -i td/test-erasure-eio/1/new.json 2026-02-21T11:22:22.372 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:22.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:22:22.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBulZlpJqSsCBAA6121BXR0zcDiZ0cug7JPwQ== --osd-uuid 7422c443-245b-4aec-b886-978573d5b07d 2026-02-21T11:22:22.403 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:22.400+0000 7f0034afca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:22.405 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:22.400+0000 7f0034afca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:22.406 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:22.400+0000 7f0034afca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:22.406 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:22.400+0000 7f0034afca40 -1 bdev(0x5593a7f7e400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:22:22.406 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:22.400+0000 7f0034afca40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:22:23.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:22:23.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:22:23.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:22:23.058 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:22:23.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:22:23.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:22:23.363 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:22:23.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:23.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:22:23.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:22:23.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:22:23.392 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:23.388+0000 7f508119ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:23.394 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:23.388+0000 7f508119ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:23.395 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:23.392+0000 7f508119ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:23.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:22:23.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:23.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:22:23.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:23.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:23.575 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:23.576 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:23.576 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:23.576 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:23.576 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:22:23.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:23.820 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:23.816+0000 7f508119ba40 -1 Falling back to public interface 2026-02-21T11:22:24.382 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:24.376+0000 7f508119ba40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:22:24.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:24.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:24.795 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:24.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:24.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:24.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:22:25.005 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1825929786,v1:127.0.0.1:6811/1825929786] [v2:127.0.0.1:6812/1825929786,v1:127.0.0.1:6813/1825929786] exists,up 7422c443-245b-4aec-b886-978573d5b07d 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:22:25.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:25.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:22:25.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:22:25.009 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:22:25.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4b702c42-d179-4d05-af88-4c1caacbc6b5 2026-02-21T11:22:25.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4b702c42-d179-4d05-af88-4c1caacbc6b5' 2026-02-21T11:22:25.009 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 4b702c42-d179-4d05-af88-4c1caacbc6b5 2026-02-21T11:22:25.010 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:22:25.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBxlZlpCO5UARAAXeMxqubaSiTGcUz9W566xw== 2026-02-21T11:22:25.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBxlZlpCO5UARAAXeMxqubaSiTGcUz9W566xw=="}' 2026-02-21T11:22:25.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4b702c42-d179-4d05-af88-4c1caacbc6b5 -i td/test-erasure-eio/2/new.json 2026-02-21T11:22:25.243 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:22:25.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:22:25.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBxlZlpCO5UARAAXeMxqubaSiTGcUz9W566xw== --osd-uuid 4b702c42-d179-4d05-af88-4c1caacbc6b5 2026-02-21T11:22:25.272 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:25.268+0000 7f826e303a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:25.273 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:25.268+0000 7f826e303a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:25.274 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:25.268+0000 7f826e303a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:25.275 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:25.272+0000 7f826e303a40 -1 bdev(0x556a1541c400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:22:25.275 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:25.272+0000 7f826e303a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:22:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:22:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:22:25.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:22:25.903 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:22:25.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:22:26.185 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:22:26.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:22:26.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:26.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:22:26.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:22:26.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:22:26.214 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:26.208+0000 7f507996fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:26.216 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:26.212+0000 7f507996fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:26.217 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:26.212+0000 7f507996fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:26.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:22:26.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:26.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:22:26.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:26.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:26.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:26.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:26.401 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:26.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:26.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:22:26.592 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:26.588+0000 7f507996fa40 -1 Falling back to public interface 2026-02-21T11:22:26.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:27.241 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:27.236+0000 7f507996fa40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:22:27.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:27.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:27.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:27.622 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:27.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:27.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:22:27.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:28.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:28.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:28.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:22:28.843 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:22:28.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:28.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:22:29.056 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3930283173,v1:127.0.0.1:6819/3930283173] [v2:127.0.0.1:6820/3930283173,v1:127.0.0.1:6821/3930283173] exists,up 4b702c42-d179-4d05-af88-4c1caacbc6b5 2026-02-21T11:22:29.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:29.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:29.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:29.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:22:29.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:22:29.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:29.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:22:29.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:22:29.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=29f317c2-5d93-471b-b0b3-81b02bb1dbb2 2026-02-21T11:22:29.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 29f317c2-5d93-471b-b0b3-81b02bb1dbb2' 2026-02-21T11:22:29.064 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 29f317c2-5d93-471b-b0b3-81b02bb1dbb2 2026-02-21T11:22:29.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:22:29.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB1lZlpqptiBBAAxp/jvfkuUHPuJDrCJSEsjw== 2026-02-21T11:22:29.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB1lZlpqptiBBAAxp/jvfkuUHPuJDrCJSEsjw=="}' 2026-02-21T11:22:29.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 29f317c2-5d93-471b-b0b3-81b02bb1dbb2 -i td/test-erasure-eio/3/new.json 2026-02-21T11:22:29.295 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:22:29.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:22:29.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB1lZlpqptiBBAAxp/jvfkuUHPuJDrCJSEsjw== --osd-uuid 29f317c2-5d93-471b-b0b3-81b02bb1dbb2 2026-02-21T11:22:29.326 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:29.320+0000 7f4b4a5e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:29.328 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:29.324+0000 7f4b4a5e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:29.329 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:29.324+0000 7f4b4a5e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:29.329 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:29.324+0000 7f4b4a5e0a40 -1 bdev(0x564838228400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:22:29.330 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:29.324+0000 7f4b4a5e0a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:22:29.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:22:29.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:22:29.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:22:29.934 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:22:29.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:22:30.238 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:22:30.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:22:30.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:30.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:22:30.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:22:30.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:22:30.256 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:30.252+0000 7ff7f4ea6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:30.258 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:30.252+0000 7ff7f4ea6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:30.259 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:30.256+0000 7ff7f4ea6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:30.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:22:30.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:30.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:22:30.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:30.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:30.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:30.460 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:30.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:30.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:30.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:22:30.660 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:30.656+0000 7ff7f4ea6a40 -1 Falling back to public interface 2026-02-21T11:22:30.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:31.218 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:31.212+0000 7ff7f4ea6a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:22:31.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:31.676 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:31.676 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:31.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:31.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:31.677 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:22:31.896 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3063653192,v1:127.0.0.1:6827/3063653192] [v2:127.0.0.1:6828/3063653192,v1:127.0.0.1:6829/3063653192] exists,up 29f317c2-5d93-471b-b0b3-81b02bb1dbb2 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:22:31.897 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:31.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:31.898 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:31.898 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:22:31.899 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:22:31.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=01c911ed-0acb-431f-ad62-583fbdfff2ab 2026-02-21T11:22:31.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 01c911ed-0acb-431f-ad62-583fbdfff2ab' 2026-02-21T11:22:31.900 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 01c911ed-0acb-431f-ad62-583fbdfff2ab 2026-02-21T11:22:31.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:22:31.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB3lZlpKH5oNhAAUAIP8D2T0KKQ0St2sIEvyw== 2026-02-21T11:22:31.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB3lZlpKH5oNhAAUAIP8D2T0KKQ0St2sIEvyw=="}' 2026-02-21T11:22:31.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 01c911ed-0acb-431f-ad62-583fbdfff2ab -i td/test-erasure-eio/4/new.json 2026-02-21T11:22:32.131 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:22:32.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:22:32.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB3lZlpKH5oNhAAUAIP8D2T0KKQ0St2sIEvyw== --osd-uuid 01c911ed-0acb-431f-ad62-583fbdfff2ab 2026-02-21T11:22:32.160 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:32.156+0000 7fcaa7cc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:32.161 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:32.156+0000 7fcaa7cc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:32.163 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:32.156+0000 7fcaa7cc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:32.163 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:32.160+0000 7fcaa7cc1a40 -1 bdev(0x555ea949a400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:22:32.164 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:32.160+0000 7fcaa7cc1a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:22:32.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:22:32.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:22:32.826 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:22:32.827 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:22:32.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:22:33.109 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:22:33.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:22:33.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:33.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:22:33.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:22:33.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:22:33.139 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:33.132+0000 7f95f8326a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:33.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:33.136+0000 7f95f8326a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:33.142 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:33.136+0000 7f95f8326a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:33.331 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:33.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:33.332 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:22:33.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:33.556 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:33.552+0000 7f95f8326a40 -1 Falling back to public interface 2026-02-21T11:22:34.124 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:34.120+0000 7f95f8326a40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:22:34.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:34.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:34.550 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:34.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:34.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:34.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:22:34.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:35.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:35.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:35.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:22:35.762 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:22:35.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:35.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:22:35.978 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2847627022,v1:127.0.0.1:6835/2847627022] [v2:127.0.0.1:6836/2847627022,v1:127.0.0.1:6837/2847627022] exists,up 01c911ed-0acb-431f-ad62-583fbdfff2ab 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:35.979 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:35.980 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:22:35.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:22:35.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:22:36.034 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:22:36.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:22:36.042 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:22:20.908+0000 7fbed2292a40 0 load: jerasure load: lrc 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-21T11:22:36.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:22:36.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-21T11:22:36.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:22:36.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-21T11:22:36.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:22:36.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:22:36.703 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:22:36.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:22:37.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:22:37.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:22:37.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:22:37.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:22:37.717 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:22:37.717 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:22:37.717 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:22:37.717 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:22:37.717 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:22:37.717 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:22:37.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:22:37.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:22:37.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:22:37.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:22:37.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:22:37.824 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:22:38.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:22:38.040 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:38.040 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:22:38.041 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:22:38.041 INFO:tasks.workunit.client.0.trial166.stderr:4' 2026-02-21T11:22:38.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:22:38.041 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:38.041 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:22:38.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803782 2026-02-21T11:22:38.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803782 2026-02-21T11:22:38.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782' 2026-02-21T11:22:38.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:38.119 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:22:38.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607557 2026-02-21T11:22:38.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607557 2026-02-21T11:22:38.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557' 2026-02-21T11:22:38.200 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:38.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:22:38.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444036 2026-02-21T11:22:38.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444036 2026-02-21T11:22:38.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-73014444036' 2026-02-21T11:22:38.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:38.281 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:22:38.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182404 2026-02-21T11:22:38.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182404 2026-02-21T11:22:38.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-73014444036 3-107374182404' 2026-02-21T11:22:38.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:22:38.360 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:22:38.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920771 2026-02-21T11:22:38.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920771 2026-02-21T11:22:38.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-73014444036 3-107374182404 4-141733920771' 2026-02-21T11:22:38.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:38.438 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803782 2026-02-21T11:22:38.438 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:38.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:22:38.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803782 2026-02-21T11:22:38.439 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:38.441 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803782 2026-02-21T11:22:38.441 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-21T11:22:38.441 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803782 2026-02-21T11:22:38.441 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:22:38.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-21T11:22:38.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:38.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607557 2026-02-21T11:22:38.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:38.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:22:38.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607557 2026-02-21T11:22:38.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:38.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607557 2026-02-21T11:22:38.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-21T11:22:38.671 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607557 2026-02-21T11:22:38.671 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:22:38.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-21T11:22:38.896 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:38.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444036 2026-02-21T11:22:38.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:38.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:22:38.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444036 2026-02-21T11:22:38.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:38.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444036 2026-02-21T11:22:38.900 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444036 2026-02-21T11:22:38.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-21T11:22:38.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:22:39.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-21T11:22:39.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:39.114 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182404 2026-02-21T11:22:39.114 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:39.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:22:39.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182404 2026-02-21T11:22:39.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:39.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182404 2026-02-21T11:22:39.118 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182404 2026-02-21T11:22:39.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-21T11:22:39.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:22:39.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-21T11:22:39.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:22:39.340 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920771 2026-02-21T11:22:39.340 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:22:39.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:22:39.342 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920771 2026-02-21T11:22:39.342 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:22:39.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920771 2026-02-21T11:22:39.344 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920771 2026-02-21T11:22:39.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920771' 2026-02-21T11:22:39.344 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:22:39.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920769 -lt 141733920771 2026-02-21T11:22:39.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:22:40.563 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:22:40.564 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:22:40.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920772 -lt 141733920771 2026-02-21T11:22:40.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:22:40.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:22:40.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:22:41.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:22:41.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:22:41.061 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:22:41.061 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:22:41.061 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:22:41.061 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:22:41.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:22:41.062 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:22:41.268 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:22:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:22:41.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:22:41.269 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:22:41.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:22:41.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:22:41.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:22:41.545 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:37.542097+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-21T11:22:36.648245+0000 0'0 2026-02-21T11:22:36.648245+0000 0 0 periodic scrub scheduled @ 2026-02-22T19:48:05.359032+0000 0 0 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.522953+0000 0'0 41:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T14:33:07.413452+0000 0 0 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.523371+0000 0'0 40:93 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T19:29:00.823301+0000 0 0 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.121901+0000 0'0 41:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T14:43:16.064889+0000 0 0 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:27.962971+0000 0'0 40:62 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T22:34:29.353650+0000 0 0 2026-02-21T11:22:41.749 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:22:41.750 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:22:41.750 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:22:41.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:22:41.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:22:41.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:22:41.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:41.787 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-21T11:22:41.787 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:22:41.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:22:41.788 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:22:41.788 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:22:42.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:22:42.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:22:42.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:22:42.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:22:42.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:22:42.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-21T11:22:42.521 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already down. 2026-02-21T11:22:42.530 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-21T11:22:42.808 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already out. 2026-02-21T11:22:42.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-21T11:22:43.027 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:22:43.027 INFO:tasks.workunit.client.0.trial166.stdout:2.0 1 0 1 0 0 4096 0 0 1 0 1 active+undersized+degraded 2026-02-21T11:22:42.191768+0000 41'1 43:27 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-21T11:22:36.648245+0000 0'0 2026-02-21T11:22:36.648245+0000 0 0 periodic scrub scheduled @ 2026-02-22T14:54:28.053877+0000 0 0 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.522953+0000 0'0 41:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T14:33:07.413452+0000 0 0 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.523371+0000 0'0 43:99 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T19:29:00.823301+0000 0 0 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.121901+0000 0'0 41:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T14:43:16.064889+0000 0 0 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:27.962971+0000 0'0 41:64 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T22:34:29.353650+0000 0 0 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:22:43.028 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:22:43.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-21T11:22:43.038 INFO:tasks.workunit.client.0.trial166.stderr:4+0 records in 2026-02-21T11:22:43.038 INFO:tasks.workunit.client.0.trial166.stderr:4+0 records out 2026-02-21T11:22:43.038 INFO:tasks.workunit.client.0.trial166.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.5432e-05 s, 62.6 MB/s 2026-02-21T11:22:43.038 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-21T11:22:43.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.257 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.356 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.448 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.449 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.499 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.640 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.904 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:43.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:43.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.098 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.551 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.672 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.776 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.825 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.944 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.944 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:44.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:44.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:22:45.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj75 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj75 2026-02-21T11:22:45.424 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:22:45.425 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-21T11:22:45.425 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-21T11:22:45.425 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:22:45.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:22:45.643 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:45.643 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:22:45.643 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:22:45.643 INFO:tasks.workunit.client.0.trial166.stderr:2147483647' 2026-02-21T11:22:45.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:22:45.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:22:45.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:22:45.645 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:22:45.645 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:22:45.645 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:22:45.645 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:22:45.646 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:45.646 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:45.646 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:45.646 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:22:45.647 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:22:45.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:22:45.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:45.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:22:45.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:22:45.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj75 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj75 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:22:45.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-21T11:22:45.771 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-21T11:22:45.771 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:2147483647' 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:22:45.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-21T11:22:45.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:22:45.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-21T11:22:45.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:22:45.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-21T11:22:45.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:22:45.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:22:45.992 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:22:45.992 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-21T11:22:45.992 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:22:45.992 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:22:45.993 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:45.993 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:45.993 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:45.993 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:22:45.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:22:46.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:22:46.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:22:46.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:22:46.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:22:46.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-21T11:22:46.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-21T11:22:46.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:22:46.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:22:46.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:22:46.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:22:46.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:22:46.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:46.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:22:46.118 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:22:46.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:22:46.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:22:46.119 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:22:46.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:22:46.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:22:46.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:22:46.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:22:46.136 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:46.132+0000 7ff122131a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:46.138 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:46.132+0000 7ff122131a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:46.138 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:46.132+0000 7ff122131a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:22:46.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:22:46.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:22:46.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:22:46.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:22:46.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:22:46.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:46.329 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:22:46.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:22:46.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:46.329 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:22:46.489 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:46.484+0000 7ff122131a40 -1 Falling back to public interface 2026-02-21T11:22:46.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:22:47.142 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:22:47.136+0000 7ff122131a40 -1 osd.2 41 log_to_monitors true 2026-02-21T11:22:47.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:22:47.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:22:47.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:22:47.541 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:22:47.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:22:47.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:22:47.753 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up out weight 0 up_from 47 up_thru 17 down_at 42 last_clean_interval [17,41) [v2:127.0.0.1:6818/1480121493,v1:127.0.0.1:6819/1480121493] [v2:127.0.0.1:6820/1480121493,v1:127.0.0.1:6821/1480121493] exists,up 4b702c42-d179-4d05-af88-4c1caacbc6b5 2026-02-21T11:22:47.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:22:47.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:22:47.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:22:47.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-21T11:22:48.017 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already in. 2026-02-21T11:22:48.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-21T11:23:03.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-21T11:23:03.030 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-21T11:23:03.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-21T11:23:03.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-21T11:23:03.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-21T11:23:03.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-21T11:23:03.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-21T11:23:03.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-21T11:23:03.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-21T11:23:03.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-21T11:23:03.246 INFO:tasks.workunit.client.0.trial166.stdout:active+recovery_unfound+degraded 2026-02-21T11:23:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-21T11:23:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-21T11:23:03.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-21T11:22:48.168397+0000 45'101 51:353 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-21T11:22:36.648245+0000 0'0 2026-02-21T11:22:36.648245+0000 0 0 no scrub is scheduled 0 0 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:48.065961+0000 0'0 51:59 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T15:19:01.746895+0000 0 0 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.523371+0000 0'0 51:113 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T19:29:00.823301+0000 0 0 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:35.121901+0000 0'0 51:41 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T14:43:16.064889+0000 0 0 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T11:22:48.066269+0000 0'0 51:101 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T11:22:18.170982+0000 0'0 2026-02-21T11:22:18.170982+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:10:23.590005+0000 0 0 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout: 2026-02-21T11:23:03.460 INFO:tasks.workunit.client.0.trial166.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T11:23:03.461 INFO:tasks.workunit.client.0.trial166.stderr:dumped pgs 2026-02-21T11:23:03.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout:{ 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout: "num_missing": 1, 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout: "num_unfound": 1, 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout: "objects": [ 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout: "oid": { 2026-02-21T11:23:03.542 INFO:tasks.workunit.client.0.trial166.stdout: "oid": "obj75", 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "key": "", 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "snapid": -2, 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "hash": 3565005477, 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "max": 0, 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "pool": 2, 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "namespace": "" 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "need": "45'76", 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "have": "0'0", 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "flags": "none", 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-21T11:23:03.543 INFO:tasks.workunit.client.0.trial166.stdout: "locations": [ 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: ] 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: "state": "NotRecovering", 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: "available_might_have_unfound": true, 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: "might_have_unfound": [], 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout: "more": false 2026-02-21T11:23:03.544 INFO:tasks.workunit.client.0.trial166.stdout:} 2026-02-21T11:23:03.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout:{ 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: "snap_trimq": "[]", 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: "snap_trimq_len": 0, 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+recovery_unfound+degraded", 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: "epoch": 51, 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.633 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: "acting_recovery_backfill": [ 2026-02-21T11:23:03.634 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "2(4)", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "info": { 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s0", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "0", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'101", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'75", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "0'0", 2026-02-21T11:23:03.635 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 101, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "51", 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 50, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:23:03.636 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 50, 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 50, 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:23:03.637 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'101", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 354, 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 51, 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+recovery_unfound+degraded", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:22:48.168397+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:22:48.168397+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:22:48.168397+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:22:48.168397+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:22:41.782741+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:22:48.066595+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:22:48.066595+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:22:48.168397+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:22:48.064632+0000", 2026-02-21T11:23:03.638 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:22:48.168397+0000", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 50, 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "0'0", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "0'0", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.639 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 0, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:23:03.640 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 413696, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 101, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 505, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 1, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 0, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 3, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:23:03.641 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 1, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 101, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 101, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 404, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 99, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 405504, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:23:03.642 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:23:03.643 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.644 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:23:03.645 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 100 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),4(2)", 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 1 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:23:03.646 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.647 INFO:tasks.workunit.client.0.trial166.stdout: "peer_info": [ 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "0(3)", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s3", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "3", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'101", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'101", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "0'0", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 101, 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:23:03.648 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 50, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 50, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 50, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.649 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'101", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 132, 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 45, 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+remapped", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.650 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:22:41.782741+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:22:42.190285+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:22:42.190224+0000", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 50, 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "0'0", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "0'0", 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:23:03.651 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 0, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:23:03.652 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T14:54:28.053877+0000", 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:23:03.653 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 413696, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 101, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 505, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 101, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 101, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 101, 2026-02-21T11:23:03.654 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 404, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:23:03.655 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:23:03.656 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.657 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 101 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.658 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:23:03.659 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "1(1)", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s1", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "1", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'101", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'101", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "0'0", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 101, 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:23:03.660 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 50, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 50, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 50, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:23:03.661 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'101", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 132, 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 45, 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+remapped", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.662 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:22:41.782741+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:22:42.190285+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:22:42.190224+0000", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 50, 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "0'0", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "0'0", 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:23:03.663 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 0, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:23:03.664 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T14:54:28.053877+0000", 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 413696, 2026-02-21T11:23:03.665 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 101, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 505, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 1, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 101, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 101, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:23:03.666 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 101, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 404, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:23:03.667 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:23:03.668 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:23:03.669 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 101 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:23:03.670 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "2(4)", 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s4", 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "4", 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'101", 2026-02-21T11:23:03.671 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "41'1", 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "0'0", 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 101, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 50, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:23:03.672 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 50, 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 50, 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'101", 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 139, 2026-02-21T11:23:03.673 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 49, 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "state": "remapped+peering", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:22:47.545243+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:22:47.443800+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:22:47.443800+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:22:47.340020+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:22:41.782741+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:22:47.545243+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:22:47.545243+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:22:47.545243+0000", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 50, 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "0'0", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "0'0", 2026-02-21T11:23:03.674 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 0, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:23:03.675 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 413696, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 101, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 505, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 1, 2026-02-21T11:23:03.676 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 101, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 101, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 404, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:23:03.677 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:23:03.678 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [], 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [], 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:23:03.679 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "peer": "4(2)", 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s2", 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "shard": "2", 2026-02-21T11:23:03.680 INFO:tasks.workunit.client.0.trial166.stdout: "last_update": "45'101", 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_complete": "45'101", 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "log_tail": "0'0", 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_user_version": 101, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill": "MAX", 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete": [], 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [], 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "history": { 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_created": 39, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "epoch_pool_created": 39, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_started": 50, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_interval_clean": 39, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_split": 0, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_marked_full": 0, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "same_up_since": 50, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "same_interval_since": 50, 2026-02-21T11:23:03.681 INFO:tasks.workunit.client.0.trial166.stdout: "same_primary_since": 39, 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "prior_readable_until_ub": 0 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "stats": { 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "version": "45'101", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "reported_seq": 132, 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "reported_epoch": 45, 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "state": "active+remapped", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_fresh": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_change": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_active": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_peered": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean": "2026-02-21T11:22:41.782741+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_active": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.682 INFO:tasks.workunit.client.0.trial166.stdout: "last_became_peered": "2026-02-21T11:22:42.191768+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_unstale": "2026-02-21T11:22:45.420373+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_undegraded": "2026-02-21T11:22:42.190285+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_fullsized": "2026-02-21T11:22:42.190224+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "mapping_epoch": 50, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "log_start": "0'0", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_start": "0'0", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "created": 39, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_clean": 40, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "parent": "0.0", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "parent_split_bits": 0, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub": "0'0", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub": "0'0", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_deep_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "last_clean_scrub_stamp": "2026-02-21T11:22:36.648245+0000", 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "objects_scrubbed": 0, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "log_size": 101, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "log_dups_size": 0, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "ondisk_log_size": 101, 2026-02-21T11:23:03.683 INFO:tasks.workunit.client.0.trial166.stdout: "stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "dirty_stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "omap_stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "hitset_bytes_stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "pin_stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "manifest_stats_invalid": false, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrimq_len": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "last_scrub_duration": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T14:54:28.053877+0000", 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "scrub_duration": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "objects_trimmed": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "snaptrim_duration": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "stat_sum": { 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes": 413696, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects": 101, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_clones": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_object_copies": 505, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_missing": 0, 2026-02-21T11:23:03.684 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_degraded": 101, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_misplaced": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_unfound": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_dirty": 101, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_whiteouts": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_read": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_read_kb": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_write": 101, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_write_kb": 404, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_scrub_errors": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_deep_scrub_errors": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_recovered": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_recovered": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_keys_recovered": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_omap": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_kb": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_kb": 0, 2026-02-21T11:23:03.685 INFO:tasks.workunit.client.0.trial166.stdout: "num_promote": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_high": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_flush_mode_low": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_some": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_evict_mode_full": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_pinned": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_legacy_snapsets": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_large_omap_objects": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_manifest": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_bytes": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_omap_keys": 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "num_objects_repaired": 0 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "up": [ 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: "acting": [ 2026-02-21T11:23:03.686 INFO:tasks.workunit.client.0.trial166.stdout: 3, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: 1, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: 4, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: 0, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: 2 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "avail_no_missing": [ 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "3(0)", 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "0(3)", 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "1(1)", 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "4(2)" 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "object_location_counts": [ 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "objects": 101 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "blocked_by": [], 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "up_primary": 3, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "acting_primary": 3, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "purged_snaps": [] 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.687 INFO:tasks.workunit.client.0.trial166.stdout: "empty": 0, 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "dne": 0, 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "incomplete": 0, 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "last_epoch_started": 51, 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "hit_set_history": { 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "current_last_update": "0'0", 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "history": [] 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "recovery_state": [ 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "name": "Started/Primary/Active", 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "enter_time": "2026-02-21T11:22:48.064334+0000", 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "might_have_unfound": [ 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "0(3)", 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "1(1)", 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:23:03.688 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "2(4)", 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "osd": "4(2)", 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "status": "already probed" 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "recovery_progress": { 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "backfill_targets": [], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "waiting_on_backfill": [], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "last_backfill_started": "MIN", 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "backfill_info": { 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "begin": "MIN", 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "end": "MIN", 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "objects": [] 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "peer_backfill_info": [], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "backfills_in_flight": [], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "recovering": [], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "pg_backend": { 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "recovery_ops": [], 2026-02-21T11:23:03.689 INFO:tasks.workunit.client.0.trial166.stdout: "read_ops": [] 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: { 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "name": "Started", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "enter_time": "2026-02-21T11:22:47.962103+0000" 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: ], 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "scrubber": { 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "active": false, 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "must_scrub": false, 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "must_deep_scrub": false, 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "must_repair": false, 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "sched_time": "2026-02-22T14:54:28.053877+0000", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "schedule": "not registered for scrubbing", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "urgency": "periodic-regular", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "shallow-target": { 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s0", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "level": "shallow", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "urgency": "periodic-regular", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "sched_time": "2026-02-22T14:54:28.053877+0000", 2026-02-21T11:23:03.690 INFO:tasks.workunit.client.0.trial166.stdout: "orig_sched_time": "2026-02-22T14:54:28.053877+0000", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "last_issue": "ok", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "forced": false, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "eligible": false, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "queued": false, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "active": false 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "deep-target": { 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "pgid": "2.0s0", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "level": "deep", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "urgency": "periodic-regular", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "sched_time": "2026-02-25T16:10:36.648245+0000", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "orig_sched_time": "2026-02-25T16:10:36.648245+0000", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "last_issue": "ok", 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "forced": false, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "eligible": false, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "queued": false, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "active": false 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: } 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: }, 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout: "agent_state": {} 2026-02-21T11:23:03.691 INFO:tasks.workunit.client.0.trial166.stdout:} 2026-02-21T11:23:03.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:23:03.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-21T11:23:03.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:23:03.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-21T11:23:03.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-21T11:23:03.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-21T11:23:03.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-21T11:23:03.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-21T11:23:03.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-21T11:23:03.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-21T11:23:03.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-21T11:23:08.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-21T11:23:08.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-21T11:23:08.962 INFO:tasks.workunit.client.0.trial166.stderr:pg has no unfound objects 2026-02-21T11:23:08.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-21T11:23:08.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:23:08.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:23:08.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:23:08.974 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:23:08.974 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:23:08.974 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:23:08.975 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:23:08.975 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:23:08.975 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:23:09.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:23:09.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:23:09.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:23:09.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:23:09.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:23:09.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:23:09.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:23:09.291 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:23:09.292 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:23:09.292 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:23:09.292 INFO:tasks.workunit.client.0.trial166.stderr:4' 2026-02-21T11:23:09.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:23:09.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:09.292 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:23:09.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803790 2026-02-21T11:23:09.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803790 2026-02-21T11:23:09.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803790' 2026-02-21T11:23:09.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:09.374 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:23:09.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607565 2026-02-21T11:23:09.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607565 2026-02-21T11:23:09.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803790 1-51539607565' 2026-02-21T11:23:09.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:09.451 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:23:09.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=201863462919 2026-02-21T11:23:09.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 201863462919 2026-02-21T11:23:09.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803790 1-51539607565 2-201863462919' 2026-02-21T11:23:09.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:09.532 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:23:09.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182412 2026-02-21T11:23:09.608 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182412 2026-02-21T11:23:09.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803790 1-51539607565 2-201863462919 3-107374182412' 2026-02-21T11:23:09.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:09.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:23:09.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920779 2026-02-21T11:23:09.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920779 2026-02-21T11:23:09.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803790 1-51539607565 2-201863462919 3-107374182412 4-141733920779' 2026-02-21T11:23:09.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:09.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803790 2026-02-21T11:23:09.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:09.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:23:09.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803790 2026-02-21T11:23:09.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:09.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803790 2026-02-21T11:23:09.692 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803790 2026-02-21T11:23:09.692 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803790' 2026-02-21T11:23:09.692 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:23:09.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803790 2026-02-21T11:23:09.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:23:10.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:23:10.907 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:23:11.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803790 2026-02-21T11:23:11.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:11.129 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607565 2026-02-21T11:23:11.129 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:11.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:23:11.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607565 2026-02-21T11:23:11.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:11.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607565 2026-02-21T11:23:11.133 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607565 2026-02-21T11:23:11.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607565' 2026-02-21T11:23:11.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:23:11.348 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607565 -lt 51539607565 2026-02-21T11:23:11.349 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:11.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-201863462919 2026-02-21T11:23:11.349 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:11.350 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:23:11.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-201863462919 2026-02-21T11:23:11.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:11.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=201863462919 2026-02-21T11:23:11.353 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 201863462919 2026-02-21T11:23:11.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 201863462919' 2026-02-21T11:23:11.354 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:23:11.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-21T11:23:11.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:11.584 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182412 2026-02-21T11:23:11.584 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:11.584 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:23:11.585 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182412 2026-02-21T11:23:11.585 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:11.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182412 2026-02-21T11:23:11.587 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182412 2026-02-21T11:23:11.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182412' 2026-02-21T11:23:11.588 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:23:11.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182412 -lt 107374182412 2026-02-21T11:23:11.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:11.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920779 2026-02-21T11:23:11.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:11.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:23:11.809 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920779 2026-02-21T11:23:11.809 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:11.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920779 2026-02-21T11:23:11.811 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 141733920779 2026-02-21T11:23:11.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920779' 2026-02-21T11:23:11.812 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:23:12.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920780 -lt 141733920779 2026-02-21T11:23:12.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:23:12.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:23:12.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:23:12.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:23:12.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:23:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:23:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:23:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:23:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:23:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:23:12.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:23:12.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:23:12.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:23:12.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:23:12.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:23:12.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:23:12.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:23:12.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:23:12.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-21T11:23:12.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-21T11:23:12.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-21T11:23:12.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-21T11:23:12.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-21T11:23:12.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-21T11:23:12.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-21T11:23:12.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-21T11:23:12.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-21T11:23:12.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-21T11:23:12.974 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-21T11:23:12.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:12.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:12.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-21T11:23:12.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-21T11:23:13.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.044 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-21T11:23:13.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-21T11:23:13.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-21T11:23:13.095 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-21T11:23:13.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.141 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-21T11:23:13.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-21T11:23:13.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-21T11:23:13.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.201 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-21T11:23:13.202 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-21T11:23:13.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-21T11:23:13.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-21T11:23:13.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-21T11:23:13.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.305 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-21T11:23:13.306 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-21T11:23:13.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-21T11:23:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-21T11:23:13.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-21T11:23:13.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-21T11:23:13.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-21T11:23:13.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-21T11:23:13.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-21T11:23:13.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-21T11:23:13.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-21T11:23:13.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-21T11:23:13.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.573 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-21T11:23:13.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-21T11:23:13.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-21T11:23:13.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-21T11:23:13.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-21T11:23:13.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-21T11:23:13.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-21T11:23:13.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-21T11:23:13.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-21T11:23:13.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-21T11:23:13.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-21T11:23:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-21T11:23:13.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-21T11:23:13.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-21T11:23:13.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-21T11:23:13.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-21T11:23:13.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-21T11:23:13.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-21T11:23:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-21T11:23:13.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-21T11:23:13.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:13.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:13.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-21T11:23:13.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-21T11:23:14.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.028 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-21T11:23:14.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.049 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-21T11:23:14.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-21T11:23:14.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-21T11:23:14.101 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-21T11:23:14.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-21T11:23:14.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-21T11:23:14.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-21T11:23:14.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.205 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-21T11:23:14.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-21T11:23:14.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.243 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-21T11:23:14.244 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-21T11:23:14.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.290 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.290 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-21T11:23:14.290 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-21T11:23:14.313 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-21T11:23:14.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-21T11:23:14.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-21T11:23:14.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-21T11:23:14.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-21T11:23:14.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.440 INFO:tasks.workunit.client.0.trial166.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-21T11:23:14.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-21T11:23:14.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.467 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-21T11:23:14.468 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-21T11:23:14.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-21T11:23:14.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-21T11:23:14.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-21T11:23:14.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-21T11:23:14.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-21T11:23:14.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-21T11:23:14.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-21T11:23:14.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-21T11:23:14.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-21T11:23:14.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-21T11:23:14.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-21T11:23:14.754 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-21T11:23:14.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-21T11:23:14.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-21T11:23:14.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-21T11:23:14.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-21T11:23:14.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-21T11:23:14.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-21T11:23:14.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-21T11:23:14.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-21T11:23:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.968 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.968 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-21T11:23:14.968 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-21T11:23:14.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:14.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-21T11:23:14.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-21T11:23:14.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-21T11:23:15.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:15.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-21T11:23:15.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:23:15.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:23:15.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:23:15.292 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:23:15.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:23:15.577 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:23:15.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:23:15.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:23:15.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:23:15.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:23:15.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:23:15.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:23:15.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:23:15.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:23:15.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:23:15.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:23:15.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:23:15.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:23:15.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:23:15.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:23:15.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:23:15.906 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:23:15.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:15.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:23:15.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:23:15.908 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:15.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:23:15.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:23:15.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:23:15.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:23:15.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:15.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:15.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:23:15.941 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:23:15.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:23:15.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:23:15.944 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:23:15.944 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:23:15.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:23:15.946 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:23:15.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:23:15.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:23:15.947 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:23:15.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:15.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:23:15.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:23:15.950 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:15.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:23:15.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:23:15.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:23:15.953 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:23:15.953 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:15.953 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:15.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:23:15.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:23:15.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:23:15.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:23:15.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:23:15.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:15.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:15.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:23:15.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:23:15.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:23:15.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:23:15.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:23:15.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:23:15.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:23:15.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:23:15.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:23:15.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:23:15.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:23:15.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:15.982 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:15.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:23:16.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:23:16.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:23:16.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:23:16.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:23:16.012 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:23:16.013 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:23:16.013 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:23:16.013 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:23:16.013 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:23:16.014 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:16.014 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:16.014 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:16.014 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:23:16.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:23:16.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:23:16.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:23:16.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:23:16.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:23:16.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:23:16.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:16.071 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:23:16.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:23:16.072 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:23:16.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:23:16.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:23:16.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:23:16.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:23:16.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:23:16.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:23:16.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:23:16.237 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:23:16.238 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:16.238 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:16.238 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:16.238 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:16.238 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:16.239 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:16.239 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:23:16.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:23:16.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:23:16.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:23:16.375 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:23:16.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:23:17.386 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:23:17.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:23:17.436 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:23:17.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:23:17.444 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:23:15.995+0000 7f0825620ec0 0 load: jerasure load: lrc 2026-02-21T11:23:17.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-21T11:23:17.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-21T11:23:17.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-21T11:23:17.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-21T11:23:17.446 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-21T11:23:17.447 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:23:17.447 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-21T11:23:17.447 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-21T11:23:17.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:17.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:23:17.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:17.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:17.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:17.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5d5c1a7b-2bc7-4f71-9962-4c18737fb77f 2026-02-21T11:23:17.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5d5c1a7b-2bc7-4f71-9962-4c18737fb77f' 2026-02-21T11:23:17.456 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 5d5c1a7b-2bc7-4f71-9962-4c18737fb77f 2026-02-21T11:23:17.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:17.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCllZlpr/F8GxAAvjQMUNSyGqCrDtQrKoLlWQ== 2026-02-21T11:23:17.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCllZlpr/F8GxAAvjQMUNSyGqCrDtQrKoLlWQ=="}' 2026-02-21T11:23:17.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5d5c1a7b-2bc7-4f71-9962-4c18737fb77f -i td/test-erasure-eio/0/new.json 2026-02-21T11:23:17.572 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:17.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:23:17.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCllZlpr/F8GxAAvjQMUNSyGqCrDtQrKoLlWQ== --osd-uuid 5d5c1a7b-2bc7-4f71-9962-4c18737fb77f 2026-02-21T11:23:17.597 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:17.591+0000 7f47598b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:17.598 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:17.595+0000 7f47598b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:17.599 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:17.595+0000 7f47598b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:17.599 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:17.595+0000 7f47598b2a40 -1 bdev(0x555b55810400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:17.600 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:17.595+0000 7f47598b2a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:23:18.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:23:18.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:18.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:23:18.267 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:23:18.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:18.383 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:23:18.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:23:18.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:18.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:18.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:18.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:18.397 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:18.391+0000 7f55d1022a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:18.399 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:18.395+0000 7f55d1022a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:18.399 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:18.395+0000 7f55d1022a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:18.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:23:18.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:18.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:23:18.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:18.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:18.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:18.537 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:18.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:18.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:18.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:23:18.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:18.808 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:18.803+0000 7f55d1022a40 -1 Falling back to public interface 2026-02-21T11:23:19.364 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:19.359+0000 7f55d1022a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:23:19.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:19.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:19.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:19.648 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:19.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:19.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:23:19.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:20.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:20.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:20.862 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:23:20.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:23:20.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:20.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:23:21.075 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1274117847,v1:127.0.0.1:6803/1274117847] [v2:127.0.0.1:6804/1274117847,v1:127.0.0.1:6805/1274117847] exists,up 5d5c1a7b-2bc7-4f71-9962-4c18737fb77f 2026-02-21T11:23:21.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:21.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:23:21.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:23:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:23:21.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:23:21.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:21.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:21.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:21.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:21.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:21.086 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:21.086 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:21.086 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:21.086 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:21.086 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:21.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:21.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:21.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:21.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:21.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:21.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:21.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:23:21.088 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:21.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=02a8b123-9005-418b-8c38-02b5749b5bec 2026-02-21T11:23:21.089 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 02a8b123-9005-418b-8c38-02b5749b5bec 2026-02-21T11:23:21.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 02a8b123-9005-418b-8c38-02b5749b5bec' 2026-02-21T11:23:21.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:21.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCplZlpjydvBRAAKwnB1O6Qx344JoaxiJ2Wdg== 2026-02-21T11:23:21.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCplZlpjydvBRAAKwnB1O6Qx344JoaxiJ2Wdg=="}' 2026-02-21T11:23:21.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 02a8b123-9005-418b-8c38-02b5749b5bec -i td/test-erasure-eio/1/new.json 2026-02-21T11:23:21.313 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:21.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:23:21.325 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCplZlpjydvBRAAKwnB1O6Qx344JoaxiJ2Wdg== --osd-uuid 02a8b123-9005-418b-8c38-02b5749b5bec 2026-02-21T11:23:21.350 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:21.343+0000 7f02a28a9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:21.351 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:21.347+0000 7f02a28a9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:21.352 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:21.347+0000 7f02a28a9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:21.353 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:21.347+0000 7f02a28a9a40 -1 bdev(0x561ca3eb6400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:21.354 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:21.347+0000 7f02a28a9a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:23:21.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:23:21.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:21.991 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:23:21.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:23:21.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:22.297 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:23:22.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:23:22.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:22.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:22.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:22.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:22.322 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:22.315+0000 7fe0d685aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:22.323 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:22.319+0000 7fe0d685aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:22.324 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:22.319+0000 7fe0d685aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:22.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:23:22.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:22.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:23:22.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:22.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:22.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:22.512 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:22.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:22.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:22.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:23:22.716 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:22.711+0000 7fe0d685aa40 -1 Falling back to public interface 2026-02-21T11:23:22.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:23.304 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:23.299+0000 7fe0d685aa40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:23:23.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:23.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:23.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:23.730 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:23.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:23.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:23:23.944 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1979543212,v1:127.0.0.1:6811/1979543212] [v2:127.0.0.1:6812/1979543212,v1:127.0.0.1:6813/1979543212] exists,up 02a8b123-9005-418b-8c38-02b5749b5bec 2026-02-21T11:23:23.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:23.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:23.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:23.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:23.946 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:23.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:23.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:23:23.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:23.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a7fe7283-817f-4627-90a1-5fb861d6effc 2026-02-21T11:23:23.949 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 a7fe7283-817f-4627-90a1-5fb861d6effc 2026-02-21T11:23:23.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 a7fe7283-817f-4627-90a1-5fb861d6effc' 2026-02-21T11:23:23.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:23.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrlZlpBNIyORAA5ZhFFNk0AIx55jfhHPoh+g== 2026-02-21T11:23:23.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrlZlpBNIyORAA5ZhFFNk0AIx55jfhHPoh+g=="}' 2026-02-21T11:23:23.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a7fe7283-817f-4627-90a1-5fb861d6effc -i td/test-erasure-eio/2/new.json 2026-02-21T11:23:24.180 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:23:24.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:23:24.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCrlZlpBNIyORAA5ZhFFNk0AIx55jfhHPoh+g== --osd-uuid a7fe7283-817f-4627-90a1-5fb861d6effc 2026-02-21T11:23:24.207 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:24.203+0000 7f18bee64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:24.209 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:24.203+0000 7f18bee64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:24.210 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:24.203+0000 7f18bee64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:24.210 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:24.207+0000 7f18bee64a40 -1 bdev(0x564df1fe0400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:24.211 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:24.207+0000 7f18bee64a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:23:24.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:23:24.849 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:24.849 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:23:24.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:23:24.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:25.146 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:23:25.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:23:25.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:25.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:25.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:25.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:25.161 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:25.155+0000 7fb6ae94ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:25.163 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:25.159+0000 7fb6ae94ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:25.164 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:25.159+0000 7fb6ae94ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:25.357 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:25.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:25.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:25.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:23:25.548 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:25.543+0000 7fb6ae94ea40 -1 Falling back to public interface 2026-02-21T11:23:25.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:26.105 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:26.099+0000 7fb6ae94ea40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:23:26.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:26.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:26.564 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:26.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:26.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:26.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:23:26.779 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/128057526,v1:127.0.0.1:6819/128057526] [v2:127.0.0.1:6820/128057526,v1:127.0.0.1:6821/128057526] exists,up a7fe7283-817f-4627-90a1-5fb861d6effc 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:26.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:26.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:26.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:26.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:23:26.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:26.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dedaf618-eba8-4c74-aea1-bf0d8bca3dec 2026-02-21T11:23:26.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 dedaf618-eba8-4c74-aea1-bf0d8bca3dec' 2026-02-21T11:23:26.783 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 dedaf618-eba8-4c74-aea1-bf0d8bca3dec 2026-02-21T11:23:26.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:26.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCulZlpLnxZLxAAWYyLKr76q4abRKspC53jiw== 2026-02-21T11:23:26.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCulZlpLnxZLxAAWYyLKr76q4abRKspC53jiw=="}' 2026-02-21T11:23:26.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dedaf618-eba8-4c74-aea1-bf0d8bca3dec -i td/test-erasure-eio/3/new.json 2026-02-21T11:23:27.022 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:23:27.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:23:27.034 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCulZlpLnxZLxAAWYyLKr76q4abRKspC53jiw== --osd-uuid dedaf618-eba8-4c74-aea1-bf0d8bca3dec 2026-02-21T11:23:27.051 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:27.047+0000 7f7b71aeaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:27.052 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:27.047+0000 7f7b71aeaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:27.053 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:27.047+0000 7f7b71aeaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:27.053 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:27.047+0000 7f7b71aeaa40 -1 bdev(0x55718757e400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:27.054 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:27.047+0000 7f7b71aeaa40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:23:27.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:23:27.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:27.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:23:27.751 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:23:27.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:28.031 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:23:28.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:23:28.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:28.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:28.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:28.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:28.048 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:28.043+0000 7f5f2228ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:28.049 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:28.043+0000 7f5f2228ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:28.050 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:28.043+0000 7f5f2228ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:28.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:23:28.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:28.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:23:28.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:28.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:28.249 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:28.249 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:28.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:28.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:28.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:23:28.392 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:28.387+0000 7f5f2228ba40 -1 Falling back to public interface 2026-02-21T11:23:28.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:29.028 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:29.023+0000 7f5f2228ba40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:23:29.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:29.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:29.458 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:29.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:29.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:29.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:23:29.669 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4154520062,v1:127.0.0.1:6827/4154520062] [v2:127.0.0.1:6828/4154520062,v1:127.0.0.1:6829/4154520062] exists,up dedaf618-eba8-4c74-aea1-bf0d8bca3dec 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:29.670 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:29.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-21T11:23:29.673 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:29.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e72af321-8ad7-4546-af12-220d279b662d 2026-02-21T11:23:29.675 INFO:tasks.workunit.client.0.trial166.stdout:add osd4 e72af321-8ad7-4546-af12-220d279b662d 2026-02-21T11:23:29.675 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 e72af321-8ad7-4546-af12-220d279b662d' 2026-02-21T11:23:29.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:29.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCxlZlpl3nOKBAANHaDUAKp/RnSDZ3QPwh6zg== 2026-02-21T11:23:29.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCxlZlpl3nOKBAANHaDUAKp/RnSDZ3QPwh6zg=="}' 2026-02-21T11:23:29.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e72af321-8ad7-4546-af12-220d279b662d -i td/test-erasure-eio/4/new.json 2026-02-21T11:23:29.908 INFO:tasks.workunit.client.0.trial166.stdout:4 2026-02-21T11:23:29.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-21T11:23:29.920 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCxlZlpl3nOKBAANHaDUAKp/RnSDZ3QPwh6zg== --osd-uuid e72af321-8ad7-4546-af12-220d279b662d 2026-02-21T11:23:29.945 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:29.939+0000 7ff17c795a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:29.946 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:29.943+0000 7ff17c795a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:29.948 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:29.943+0000 7ff17c795a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:29.948 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:29.943+0000 7ff17c795a40 -1 bdev(0x5637382b4400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:29.948 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:29.943+0000 7ff17c795a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-21T11:23:30.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-21T11:23:30.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:30.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-21T11:23:30.617 INFO:tasks.workunit.client.0.trial166.stdout:adding osd4 key to auth repository 2026-02-21T11:23:30.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:30.893 INFO:tasks.workunit.client.0.trial166.stdout:start osd.4 2026-02-21T11:23:30.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-21T11:23:30.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:30.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:30.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:30.894 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:30.917 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:30.911+0000 7f0c7525fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:30.919 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:30.915+0000 7f0c7525fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:30.920 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:30.915+0000 7f0c7525fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:31.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-21T11:23:31.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:31.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-21T11:23:31.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:31.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:31.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:31.106 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:31.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:31.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:31.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:23:31.296 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:31.291+0000 7f0c7525fa40 -1 Falling back to public interface 2026-02-21T11:23:31.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:32.000 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:31.995+0000 7f0c7525fa40 -1 osd.4 0 log_to_monitors true 2026-02-21T11:23:32.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:32.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:32.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:32.319 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:32.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:32.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:23:32.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:33.555 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:23:33.556 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:23:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:33.556 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-21T11:23:33.764 INFO:tasks.workunit.client.0.trial166.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2899408380,v1:127.0.0.1:6835/2899408380] [v2:127.0.0.1:6836/2899408380,v1:127.0.0.1:6837/2899408380] exists,up e72af321-8ad7-4546-af12-220d279b662d 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-21T11:23:33.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:33.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:33.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:33.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:33.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:33.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:33.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-21T11:23:33.771 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:33.771 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b58e099f-bb41-402d-b8cd-c8e22bf5424a 2026-02-21T11:23:33.772 INFO:tasks.workunit.client.0.trial166.stdout:add osd5 b58e099f-bb41-402d-b8cd-c8e22bf5424a 2026-02-21T11:23:33.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 b58e099f-bb41-402d-b8cd-c8e22bf5424a' 2026-02-21T11:23:33.772 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:33.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC1lZlp40RxLhAAFinO0mq4LkXzB2PJrWLlhQ== 2026-02-21T11:23:33.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC1lZlp40RxLhAAFinO0mq4LkXzB2PJrWLlhQ=="}' 2026-02-21T11:23:33.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b58e099f-bb41-402d-b8cd-c8e22bf5424a -i td/test-erasure-eio/5/new.json 2026-02-21T11:23:34.003 INFO:tasks.workunit.client.0.trial166.stdout:5 2026-02-21T11:23:34.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-21T11:23:34.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC1lZlp40RxLhAAFinO0mq4LkXzB2PJrWLlhQ== --osd-uuid b58e099f-bb41-402d-b8cd-c8e22bf5424a 2026-02-21T11:23:34.030 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.023+0000 7fbed4e62a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:34.031 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.027+0000 7fbed4e62a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:34.032 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.027+0000 7fbed4e62a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:34.033 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.027+0000 7fbed4e62a40 -1 bdev(0x55c75bb06400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:34.033 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.027+0000 7fbed4e62a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-21T11:23:34.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-21T11:23:34.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:34.698 INFO:tasks.workunit.client.0.trial166.stdout:adding osd5 key to auth repository 2026-02-21T11:23:34.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-21T11:23:34.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:34.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-21T11:23:34.987 INFO:tasks.workunit.client.0.trial166.stdout:start osd.5 2026-02-21T11:23:34.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:34.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:34.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:34.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:35.004 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.999+0000 7fb8e97d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:35.006 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.999+0000 7fb8e97d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:35.007 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:34.999+0000 7fb8e97d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:35.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:35.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:23:35.360 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:35.355+0000 7fb8e97d7a40 -1 Falling back to public interface 2026-02-21T11:23:35.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:36.001 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:35.995+0000 7fb8e97d7a40 -1 osd.5 0 log_to_monitors true 2026-02-21T11:23:36.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:36.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:36.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:36.432 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:36.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:36.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-21T11:23:36.638 INFO:tasks.workunit.client.0.trial166.stdout:osd.5 up in weight 1 up_from 43 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/680496190,v1:127.0.0.1:6843/680496190] [v2:127.0.0.1:6844/680496190,v1:127.0.0.1:6845/680496190] exists,up b58e099f-bb41-402d-b8cd-c8e22bf5424a 2026-02-21T11:23:36.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:36.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:36.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-21T11:23:36.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:36.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8539f7c0-009b-431a-9ec8-3b696abc691c 2026-02-21T11:23:36.645 INFO:tasks.workunit.client.0.trial166.stdout:add osd6 8539f7c0-009b-431a-9ec8-3b696abc691c 2026-02-21T11:23:36.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 8539f7c0-009b-431a-9ec8-3b696abc691c' 2026-02-21T11:23:36.646 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:36.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC4lZlpNn/qJhAA4hNW4goFUqG89uFUaomyxg== 2026-02-21T11:23:36.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC4lZlpNn/qJhAA4hNW4goFUqG89uFUaomyxg=="}' 2026-02-21T11:23:36.656 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8539f7c0-009b-431a-9ec8-3b696abc691c -i td/test-erasure-eio/6/new.json 2026-02-21T11:23:36.871 INFO:tasks.workunit.client.0.trial166.stdout:6 2026-02-21T11:23:36.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-21T11:23:36.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC4lZlpNn/qJhAA4hNW4goFUqG89uFUaomyxg== --osd-uuid 8539f7c0-009b-431a-9ec8-3b696abc691c 2026-02-21T11:23:36.908 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:36.903+0000 7f9617573a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:36.910 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:36.903+0000 7f9617573a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:36.911 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:36.907+0000 7f9617573a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:36.912 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:36.907+0000 7f9617573a40 -1 bdev(0x5572bcaa8400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:36.912 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:36.907+0000 7f9617573a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-21T11:23:37.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-21T11:23:37.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:37.587 INFO:tasks.workunit.client.0.trial166.stdout:adding osd6 key to auth repository 2026-02-21T11:23:37.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-21T11:23:37.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:37.869 INFO:tasks.workunit.client.0.trial166.stdout:start osd.6 2026-02-21T11:23:37.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-21T11:23:37.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:37.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:37.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:37.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:37.894 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:37.887+0000 7f5fdc83fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:37.896 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:37.891+0000 7f5fdc83fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:37.897 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:37.891+0000 7f5fdc83fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:38.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:38.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:23:38.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:38.360 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:38.355+0000 7f5fdc83fa40 -1 Falling back to public interface 2026-02-21T11:23:38.963 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:38.959+0000 7f5fdc83fa40 -1 osd.6 0 log_to_monitors true 2026-02-21T11:23:39.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:39.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:39.297 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:39.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:39.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:39.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:23:39.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:40.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:40.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:40.567 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:23:40.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:23:40.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:40.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-21T11:23:40.771 INFO:tasks.workunit.client.0.trial166.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/264975885,v1:127.0.0.1:6851/264975885] [v2:127.0.0.1:6852/264975885,v1:127.0.0.1:6853/264975885] exists,up 8539f7c0-009b-431a-9ec8-3b696abc691c 2026-02-21T11:23:40.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:40.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:40.772 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:40.772 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:23:40.772 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:23:40.772 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:23:40.773 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:40.773 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:40.773 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:40.773 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:23:40.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:23:40.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:23:40.838 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:23:40.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:23:18.807+0000 7f55d1022a40 0 load: jerasure load: lrc 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:23:40.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-21T11:23:41.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:23:41.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:23:41.512 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:23:41.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:23:42.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:23:42.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:23:42.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:23:42.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:23:42.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:23:42.526 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:23:42.526 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:23:42.526 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:23:42.526 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:23:42.526 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:23:42.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:23:42.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:23:42.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:23:42.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:23:42.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:23:42.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:23:42.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:23:42.847 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:23:42.847 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:23:42.847 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:23:42.847 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:23:42.847 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:23:42.848 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:23:42.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:23:42.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:42.848 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:23:42.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803783 2026-02-21T11:23:42.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803783 2026-02-21T11:23:42.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783' 2026-02-21T11:23:42.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:42.929 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:23:43.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574854 2026-02-21T11:23:43.008 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574854 2026-02-21T11:23:43.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574854' 2026-02-21T11:23:43.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:43.009 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:23:43.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378630 2026-02-21T11:23:43.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378630 2026-02-21T11:23:43.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-81604378630' 2026-02-21T11:23:43.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:43.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:23:43.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116997 2026-02-21T11:23:43.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116997 2026-02-21T11:23:43.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-81604378630 3-115964116997' 2026-02-21T11:23:43.165 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:43.165 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:23:43.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=150323855365 2026-02-21T11:23:43.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 150323855365 2026-02-21T11:23:43.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-81604378630 3-115964116997 4-150323855365' 2026-02-21T11:23:43.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:43.246 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:23:43.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593732 2026-02-21T11:23:43.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593732 2026-02-21T11:23:43.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-81604378630 3-115964116997 4-150323855365 5-184683593732' 2026-02-21T11:23:43.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:43.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:23:43.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=223338299396 2026-02-21T11:23:43.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 223338299396 2026-02-21T11:23:43.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-81604378630 3-115964116997 4-150323855365 5-184683593732 6-223338299396' 2026-02-21T11:23:43.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:43.405 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803783 2026-02-21T11:23:43.405 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:43.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:23:43.407 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803783 2026-02-21T11:23:43.407 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:43.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803783 2026-02-21T11:23:43.409 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803783 2026-02-21T11:23:43.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-21T11:23:43.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:23:43.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-21T11:23:43.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:23:44.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:23:44.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:23:44.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-21T11:23:44.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:44.852 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574854 2026-02-21T11:23:44.852 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:44.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:23:44.854 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574854 2026-02-21T11:23:44.854 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:44.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574854 2026-02-21T11:23:44.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-21T11:23:44.856 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574854 2026-02-21T11:23:44.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:23:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574855 -lt 55834574854 2026-02-21T11:23:45.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:45.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378630 2026-02-21T11:23:45.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:45.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:23:45.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378630 2026-02-21T11:23:45.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:45.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378630 2026-02-21T11:23:45.067 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378630 2026-02-21T11:23:45.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378630' 2026-02-21T11:23:45.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:23:45.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378630 -lt 81604378630 2026-02-21T11:23:45.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:45.284 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116997 2026-02-21T11:23:45.284 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:45.285 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:23:45.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116997 2026-02-21T11:23:45.286 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:45.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116997 2026-02-21T11:23:45.289 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 115964116997 2026-02-21T11:23:45.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-21T11:23:45.290 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:23:45.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-21T11:23:45.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:45.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-150323855365 2026-02-21T11:23:45.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:45.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:23:45.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-150323855365 2026-02-21T11:23:45.506 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:45.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=150323855365 2026-02-21T11:23:45.508 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 150323855365 2026-02-21T11:23:45.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 150323855365' 2026-02-21T11:23:45.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:23:45.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-21T11:23:45.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:45.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-184683593732 2026-02-21T11:23:45.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:45.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:23:45.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-184683593732 2026-02-21T11:23:45.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:45.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593732 2026-02-21T11:23:45.732 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 184683593732 2026-02-21T11:23:45.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-21T11:23:45.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:23:45.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-21T11:23:45.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:45.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-223338299396 2026-02-21T11:23:45.950 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:45.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-21T11:23:45.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-223338299396 2026-02-21T11:23:45.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:45.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=223338299396 2026-02-21T11:23:45.953 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.6 seq 223338299396 2026-02-21T11:23:45.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 223338299396' 2026-02-21T11:23:45.954 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-21T11:23:46.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 223338299396 -lt 223338299396 2026-02-21T11:23:46.177 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:23:46.178 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:23:46.178 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:23:46.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:23:46.463 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:23:46.463 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:23:46.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:23:46.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:23:46.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:23:46.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:23:46.464 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:23:46.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:23:46.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:23:46.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:23:46.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:23:46.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:23:46.971 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:23:46.972 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:23:46.972 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:23:47.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:23:47.189 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:23:47.190 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:23:47.191 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:23:47.191 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:23:47.191 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:23:47.191 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:47.191 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:47.191 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:47.192 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:23:47.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:23:47.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:23:47.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:23:47.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:47.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:23:47.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:23:47.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-21T11:23:47.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-21T11:23:47.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:23:47.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-21T11:23:47.315 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-21T11:23:47.315 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:23:47.534 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:23:47.534 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:23:47.534 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:23:47.534 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:23:47.534 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:23:47.534 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:23:47.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:23:47.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:23:47.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:23:47.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:23:47.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-21T11:23:48.050 INFO:tasks.workunit.client.0.trial166.stderr:osd.6 is already down. 2026-02-21T11:23:48.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-21T11:23:48.306 INFO:tasks.workunit.client.0.trial166.stderr:osd.6 is already out. 2026-02-21T11:23:48.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-21T11:23:48.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:23:48.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:23:48.318 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:23:48.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:23:48.318 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:23:48.319 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:23:48.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:23:48.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:23:48.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:23:48.422 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:23:48.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:23:48.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:23:48.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:23:48.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:23:48.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:3 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:4 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:5 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:6' 2026-02-21T11:23:48.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:23:48.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:48.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:23:48.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803786 2026-02-21T11:23:48.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803786 2026-02-21T11:23:48.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786' 2026-02-21T11:23:48.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:48.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:23:48.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574857 2026-02-21T11:23:48.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574857 2026-02-21T11:23:48.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857' 2026-02-21T11:23:48.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:48.801 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:23:48.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378633 2026-02-21T11:23:48.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378633 2026-02-21T11:23:48.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-81604378633' 2026-02-21T11:23:48.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:48.881 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:23:48.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117000 2026-02-21T11:23:48.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117000 2026-02-21T11:23:48.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-81604378633 3-115964117000' 2026-02-21T11:23:48.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:48.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-21T11:23:49.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=150323855368 2026-02-21T11:23:49.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 150323855368 2026-02-21T11:23:49.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-81604378633 3-115964117000 4-150323855368' 2026-02-21T11:23:49.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:49.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-21T11:23:49.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593735 2026-02-21T11:23:49.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593735 2026-02-21T11:23:49.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-81604378633 3-115964117000 4-150323855368 5-184683593735' 2026-02-21T11:23:49.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:23:49.117 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-21T11:23:49.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-21T11:23:49.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-21T11:23:49.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-21T11:23:49.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-21T11:23:49.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:49.178 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803786 2026-02-21T11:23:49.178 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:49.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:23:49.180 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803786 2026-02-21T11:23:49.180 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:49.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803786 2026-02-21T11:23:49.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-21T11:23:49.182 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803786 2026-02-21T11:23:49.182 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:23:49.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-21T11:23:49.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:23:50.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:23:50.401 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:23:50.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803786 2026-02-21T11:23:50.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:50.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574857 2026-02-21T11:23:50.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:50.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:23:50.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574857 2026-02-21T11:23:50.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:50.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574857 2026-02-21T11:23:50.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-21T11:23:50.620 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574857 2026-02-21T11:23:50.620 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:23:50.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-21T11:23:50.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:50.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378633 2026-02-21T11:23:50.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:50.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:23:50.844 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378633 2026-02-21T11:23:50.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:50.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378633 2026-02-21T11:23:50.846 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378633 2026-02-21T11:23:50.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378633' 2026-02-21T11:23:50.846 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:23:51.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378633 -lt 81604378633 2026-02-21T11:23:51.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:51.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117000 2026-02-21T11:23:51.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:51.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:23:51.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117000 2026-02-21T11:23:51.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:51.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117000 2026-02-21T11:23:51.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117000' 2026-02-21T11:23:51.070 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 115964117000 2026-02-21T11:23:51.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:23:51.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117001 -lt 115964117000 2026-02-21T11:23:51.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:51.291 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-150323855368 2026-02-21T11:23:51.292 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:51.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-21T11:23:51.293 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-150323855368 2026-02-21T11:23:51.294 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:51.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=150323855368 2026-02-21T11:23:51.295 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.4 seq 150323855368 2026-02-21T11:23:51.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 150323855368' 2026-02-21T11:23:51.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-21T11:23:51.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 150323855368 -lt 150323855368 2026-02-21T11:23:51.506 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:23:51.507 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-184683593735 2026-02-21T11:23:51.507 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:23:51.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-21T11:23:51.508 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-184683593735 2026-02-21T11:23:51.509 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:23:51.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593735 2026-02-21T11:23:51.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-21T11:23:51.510 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.5 seq 184683593735 2026-02-21T11:23:51.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-21T11:23:51.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-21T11:23:51.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:23:51.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:23:51.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:23:52.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:23:52.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:23:52.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:23:52.029 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:23:52.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:23:52.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:23:52.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:23:52.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:23:52.236 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:23:52.237 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:23:52.237 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:23:52.237 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:23:52.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:23:52.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:23:52.520 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-21T11:23:52.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:23:52.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:23:52.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:23:52.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:23:52.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:23:52.805 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:23:52.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:23:53.094 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:23:53.104 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:23:53.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:23:53.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:23:53.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:23:53.218 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:23:53.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:23:53.220 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:23:53.220 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:23:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:23:53.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:23:53.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:53.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:23:53.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:23:53.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:23:53.245 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:23:53.245 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.245 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.245 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:23:53.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:23:53.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:23:53.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:23:53.248 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:23:53.250 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:23:53.250 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:23:53.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:23:53.252 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:23:53.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:23:53.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:23:53.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:23:53.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:53.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:23:53.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:23:53.256 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:23:53.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:23:53.257 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:23:53.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:23:53.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:23:53.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:23:53.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:23:53.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:23:53.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:23:53.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:23:53.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:23:53.263 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:23:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:23:53.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:23:53.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:53.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:53.287 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:53.288 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.288 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.288 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:53.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:23:53.317 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:23:53.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:23:53.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:23:53.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:23:53.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.319 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.320 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:23:53.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:23:53.320 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:23:53.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:53.379 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.380 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.380 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:23:53.380 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:23:53.380 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:23:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:23:53.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:23:53.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:23:53.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:23:53.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:23:53.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:23:53.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:53.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:53.549 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:23:53.550 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:23:53.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:23:53.568 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:23:53.672 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:23:53.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:23:54.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:23:54.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:23:54.682 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:23:54.682 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:23:54.682 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:54.682 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:54.683 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:23:54.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:23:54.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:23:54.732 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:23:54.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:23:54.739 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:23:53.303+0000 7f2998806ec0 0 load: jerasure load: lrc 2026-02-21T11:23:54.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-21T11:23:54.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-21T11:23:54.741 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-21T11:23:54.741 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:23:54.741 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:23:54.741 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:23:54.741 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:54.746 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:54.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:54.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:54.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:23:54.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:54.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=44e4725c-c9dc-41a0-9aa6-e72e9819b555 2026-02-21T11:23:54.749 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 44e4725c-c9dc-41a0-9aa6-e72e9819b555 2026-02-21T11:23:54.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 44e4725c-c9dc-41a0-9aa6-e72e9819b555' 2026-02-21T11:23:54.749 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:54.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDKlZlpeEUeLRAADu7DT/NegsDip5x4t/GQrA== 2026-02-21T11:23:54.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDKlZlpeEUeLRAADu7DT/NegsDip5x4t/GQrA=="}' 2026-02-21T11:23:54.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 44e4725c-c9dc-41a0-9aa6-e72e9819b555 -i td/test-erasure-eio/0/new.json 2026-02-21T11:23:54.871 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:54.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:23:54.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKlZlpeEUeLRAADu7DT/NegsDip5x4t/GQrA== --osd-uuid 44e4725c-c9dc-41a0-9aa6-e72e9819b555 2026-02-21T11:23:54.897 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:54.891+0000 7f0b6cb7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:54.898 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:54.895+0000 7f0b6cb7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:54.899 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:54.895+0000 7f0b6cb7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:54.900 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:54.895+0000 7f0b6cb7da40 -1 bdev(0x561444ef8400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:54.900 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:54.895+0000 7f0b6cb7da40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:23:55.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:23:55.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:55.581 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:23:55.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:23:55.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:55.731 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:23:55.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:23:55.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:55.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:55.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:55.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:55.746 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:55.739+0000 7f7ecab50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:55.747 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:55.743+0000 7f7ecab50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:55.748 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:55.743+0000 7f7ecab50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:55.842 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:55.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:55.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:55.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:23:56.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:56.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:56.135+0000 7f7ecab50a40 -1 Falling back to public interface 2026-02-21T11:23:56.776 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:56.771+0000 7f7ecab50a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:23:57.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:57.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:57.059 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:23:57.060 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:57.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:57.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:23:57.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:23:58.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:23:58.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:58.283 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:23:58.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:23:58.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:58.284 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:23:58.497 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2479197119,v1:127.0.0.1:6803/2479197119] [v2:127.0.0.1:6804/2479197119,v1:127.0.0.1:6805/2479197119] exists,up 44e4725c-c9dc-41a0-9aa6-e72e9819b555 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:23:58.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:23:58.502 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:23:58.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:23:58.504 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:23:58.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:23:58.505 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:23:58.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f737fd3b-40c0-46cf-b0f1-4873b6311b7c' 2026-02-21T11:23:58.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:23:58.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDOlZlphVOKHhAAFAiZW0cFU/ef7NUf5OCAHw== 2026-02-21T11:23:58.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDOlZlphVOKHhAAFAiZW0cFU/ef7NUf5OCAHw=="}' 2026-02-21T11:23:58.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f737fd3b-40c0-46cf-b0f1-4873b6311b7c -i td/test-erasure-eio/1/new.json 2026-02-21T11:23:58.735 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:23:58.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:23:58.746 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDOlZlphVOKHhAAFAiZW0cFU/ef7NUf5OCAHw== --osd-uuid f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:23:58.763 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:58.759+0000 7fb9c3e41a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:58.765 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:58.759+0000 7fb9c3e41a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:58.766 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:58.759+0000 7fb9c3e41a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:58.766 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:58.763+0000 7fb9c3e41a40 -1 bdev(0x5593a9f6e400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:23:58.766 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:58.763+0000 7fb9c3e41a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:23:59.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:23:59.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:23:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:23:59.402 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:23:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:23:59.692 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:23:59.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:23:59.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:23:59.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:23:59.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:23:59.693 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:23:59.717 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:59.711+0000 7f929cfd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:59.718 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:59.715+0000 7f929cfd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:59.720 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:23:59.715+0000 7f929cfd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:23:59.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:23:59.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:23:59.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:23:59.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:23:59.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:23:59.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:23:59.909 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:23:59.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:23:59.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:23:59.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:00.088 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:00.083+0000 7f929cfd1a40 -1 Falling back to public interface 2026-02-21T11:24:00.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:00.749 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:00.743+0000 7f929cfd1a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:24:01.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:01.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:01.114 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:24:01.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:01.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:01.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:01.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:02.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:02.326 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:02.327 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:24:02.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:24:02.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:02.327 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:02.545 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/445307212,v1:127.0.0.1:6811/445307212] [v2:127.0.0.1:6812/445307212,v1:127.0.0.1:6813/445307212] exists,up f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:02.546 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:24:02.547 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:24:02.548 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:24:02.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4c5b12dc-a105-4f3d-a784-d9d19de8b4ff 2026-02-21T11:24:02.549 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4c5b12dc-a105-4f3d-a784-d9d19de8b4ff' 2026-02-21T11:24:02.549 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 4c5b12dc-a105-4f3d-a784-d9d19de8b4ff 2026-02-21T11:24:02.550 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:24:02.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDSlZlpdeFxIRAAr7svKXv9u6f88YjBfuHRUg== 2026-02-21T11:24:02.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDSlZlpdeFxIRAAr7svKXv9u6f88YjBfuHRUg=="}' 2026-02-21T11:24:02.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4c5b12dc-a105-4f3d-a784-d9d19de8b4ff -i td/test-erasure-eio/2/new.json 2026-02-21T11:24:02.785 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:24:02.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:24:02.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDSlZlpdeFxIRAAr7svKXv9u6f88YjBfuHRUg== --osd-uuid 4c5b12dc-a105-4f3d-a784-d9d19de8b4ff 2026-02-21T11:24:02.821 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:02.815+0000 7f7773b1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:02.822 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:02.819+0000 7f7773b1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:02.823 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:02.819+0000 7f7773b1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:02.824 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:02.819+0000 7f7773b1ba40 -1 bdev(0x5569cbf9a400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:24:02.824 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:02.819+0000 7f7773b1ba40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:24:03.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:24:03.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:24:03.475 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:24:03.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:24:03.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:24:03.763 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:24:03.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:24:03.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:03.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:24:03.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:24:03.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:24:03.780 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:03.775+0000 7fcbdaf71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:03.781 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:03.775+0000 7fcbdaf71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:03.782 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:03.775+0000 7fcbdaf71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:03.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:24:03.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:03.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:24:03.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:03.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:03.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:03.979 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:24:03.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:03.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:03.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:24:04.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:04.260 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:04.255+0000 7fcbdaf71a40 -1 Falling back to public interface 2026-02-21T11:24:04.814 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:04.807+0000 7fcbdaf71a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:24:05.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:05.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:05.199 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:24:05.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:05.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:05.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:24:05.421 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2226321020,v1:127.0.0.1:6819/2226321020] [v2:127.0.0.1:6820/2226321020,v1:127.0.0.1:6821/2226321020] exists,up 4c5b12dc-a105-4f3d-a784-d9d19de8b4ff 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:24:05.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:24:05.426 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:05.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:05.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:05.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:05.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:05.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:24:05.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:24:05.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:24:05.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:24:05.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:05.429 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:05.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 6a69993f-bdb3-4e82-9c4a-b67aea3d305a' 2026-02-21T11:24:05.430 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:24:05.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDVlZlppJv2GRAABJdTokQesOCU6a0R5A1JQQ== 2026-02-21T11:24:05.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDVlZlppJv2GRAABJdTokQesOCU6a0R5A1JQQ=="}' 2026-02-21T11:24:05.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6a69993f-bdb3-4e82-9c4a-b67aea3d305a -i td/test-erasure-eio/3/new.json 2026-02-21T11:24:05.656 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:24:05.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:24:05.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDVlZlppJv2GRAABJdTokQesOCU6a0R5A1JQQ== --osd-uuid 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:05.683 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:05.679+0000 7fce9e95ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:05.685 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:05.679+0000 7fce9e95ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:05.686 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:05.679+0000 7fce9e95ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:05.686 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:05.679+0000 7fce9e95ba40 -1 bdev(0x562d68412400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:24:05.687 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:05.683+0000 7fce9e95ba40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:24:06.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:24:06.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:24:06.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:24:06.354 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:24:06.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:24:06.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:24:06.659 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:24:06.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:06.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:24:06.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:24:06.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:24:06.675 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:06.671+0000 7efe6eb50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:06.677 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:06.671+0000 7efe6eb50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:06.678 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:06.671+0000 7efe6eb50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:06.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:06.877 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:24:06.877 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:06.877 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:07.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:07.108 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:07.103+0000 7efe6eb50a40 -1 Falling back to public interface 2026-02-21T11:24:07.732 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:07.727+0000 7efe6eb50a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:24:08.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:08.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:08.089 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:24:08.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:08.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:08.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:08.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:09.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:09.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:09.301 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:24:09.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:24:09.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:09.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:09.511 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2854441521,v1:127.0.0.1:6827/2854441521] [v2:127.0.0.1:6828/2854441521,v1:127.0.0.1:6829/2854441521] exists,up 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:09.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:09.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:09.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:09.514 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:24:09.514 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:24:09.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:24:09.515 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:24:09.515 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:09.515 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:09.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:24:09.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:24:09.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:24:09.575 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:24:09.580 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:24:09.582 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:23:56.139+0000 7f7ecab50a40 0 load: jerasure load: lrc 2026-02-21T11:24:09.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-21T11:24:09.582 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:24:09.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:24:09.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:24:09.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:24:10.226 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:24:10.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:24:11.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:24:11.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:11.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:11.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:11.239 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:11.240 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:11.241 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:11.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:11.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:11.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:11.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:11.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:11.352 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:11.352 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:11.352 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:11.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:11.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:11.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-21T11:24:11.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-21T11:24:11.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-21T11:24:11.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:11.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:11.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-21T11:24:11.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-21T11:24:11.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-21T11:24:11.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:11.725 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:11.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411333 2026-02-21T11:24:11.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411333 2026-02-21T11:24:11.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411333' 2026-02-21T11:24:11.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:11.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149700 2026-02-21T11:24:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149700 2026-02-21T11:24:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411333 3-111669149700' 2026-02-21T11:24:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:11.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-21T11:24:11.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:11.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:11.886 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-21T11:24:11.886 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:11.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-21T11:24:11.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-21T11:24:11.888 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803781 2026-02-21T11:24:11.888 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:12.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-21T11:24:12.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:12.111 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-21T11:24:12.111 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:12.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:12.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-21T11:24:12.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:12.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-21T11:24:12.114 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574852 2026-02-21T11:24:12.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-21T11:24:12.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:12.338 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-21T11:24:12.338 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:12.339 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411333 2026-02-21T11:24:12.339 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:12.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:12.340 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411333 2026-02-21T11:24:12.340 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:12.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411333 2026-02-21T11:24:12.342 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-21T11:24:12.342 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 77309411333 2026-02-21T11:24:12.343 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:12.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411331 -lt 77309411333 2026-02-21T11:24:12.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:13.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:13.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:13.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411331 -lt 77309411333 2026-02-21T11:24:13.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:14.778 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:24:14.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:14.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-21T11:24:14.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:14.997 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149700 2026-02-21T11:24:14.998 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:14.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:15.000 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149700 2026-02-21T11:24:15.000 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:15.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149700 2026-02-21T11:24:15.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-21T11:24:15.001 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149700 2026-02-21T11:24:15.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:15.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-21T11:24:15.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:15.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:15.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:15.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:15.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:15.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:15.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:15.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74328-0-10 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74328-0-10 2026-02-21T11:24:15.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74328-0-10 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:24:15.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74328-0-10 td/test-erasure-eio/ORIGINAL 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74328-0-10 td/test-erasure-eio 0 10 set 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-0-10 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-0-10 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:24:16.018 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-0-10 2026-02-21T11:24:16.019 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-0-10 2026-02-21T11:24:16.019 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-21T11:24:16.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:24:16.526 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:24:16.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:24:16.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-21T11:24:16.536 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:10 bytes copied, 3.1108e-05 s, 321 kB/s 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-21T11:24:16.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:16.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-21T11:24:16.538 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:16.538 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:16.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:16.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:16.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:16.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:16.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:17.119 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:17.120 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:17.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:24:17.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T11:24:17.122 INFO:tasks.workunit.client.0.trial166.stderr:start osd.3 2026-02-21T11:24:17.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:17.123 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-21T11:24:17.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T11:24:17.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:17.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:17.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:17.144 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:17.138+0000 7fe3e17aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:17.145 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:17.138+0000 7fe3e17aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:17.146 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:17.142+0000 7fe3e17aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:17.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:17.557 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:17.572 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:17.566+0000 7fe3e17aea40 -1 Falling back to public interface 2026-02-21T11:24:18.181 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:18.174+0000 7fe3e17aea40 -1 osd.3 35 log_to_monitors true 2026-02-21T11:24:18.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:18.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:18.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:18.559 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:18.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:18.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:18.766 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [26,35) [v2:127.0.0.1:6826/228007295,v1:127.0.0.1:6827/228007295] [v2:127.0.0.1:6828/228007295,v1:127.0.0.1:6829/228007295] exists,up 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:18.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:18.768 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:18.768 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:18.768 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:18.768 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:18.768 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:18.768 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:18.877 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:18.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:18.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:18.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:18.878 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:18.878 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:19.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:19.100 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:19.100 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:19.100 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:19.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:19.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:19.100 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:19.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803785 2026-02-21T11:24:19.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803785 2026-02-21T11:24:19.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785' 2026-02-21T11:24:19.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:19.177 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:19.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574856 2026-02-21T11:24:19.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574856 2026-02-21T11:24:19.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-55834574856' 2026-02-21T11:24:19.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:19.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:19.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411336 2026-02-21T11:24:19.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411336 2026-02-21T11:24:19.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-55834574856 2-77309411336' 2026-02-21T11:24:19.338 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:19.338 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:19.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724547 2026-02-21T11:24:19.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724547 2026-02-21T11:24:19.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-55834574856 2-77309411336 3-167503724547' 2026-02-21T11:24:19.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:19.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803785 2026-02-21T11:24:19.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:19.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:19.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803785 2026-02-21T11:24:19.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:19.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803785 2026-02-21T11:24:19.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-21T11:24:19.420 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803785 2026-02-21T11:24:19.420 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:19.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-21T11:24:19.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:20.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:20.637 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:20.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-21T11:24:20.856 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:20.857 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574856 2026-02-21T11:24:20.857 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:20.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:20.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574856 2026-02-21T11:24:20.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:20.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574856 2026-02-21T11:24:20.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-21T11:24:20.861 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 55834574856 2026-02-21T11:24:20.861 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:21.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-21T11:24:21.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:21.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411336 2026-02-21T11:24:21.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:21.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:21.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411336 2026-02-21T11:24:21.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:21.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411336 2026-02-21T11:24:21.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-21T11:24:21.086 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411336 2026-02-21T11:24:21.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:21.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-21T11:24:21.292 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:21.293 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-167503724547 2026-02-21T11:24:21.293 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:21.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:21.296 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-167503724547 2026-02-21T11:24:21.296 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:21.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724547 2026-02-21T11:24:21.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 167503724547' 2026-02-21T11:24:21.297 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 167503724547 2026-02-21T11:24:21.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:21.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-21T11:24:21.513 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:21.514 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:21.514 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:21.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:21.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:22.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:22.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:22.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:22.014 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:22.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:22.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:22.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:22.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:24:22.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:24:22.617 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-0-10 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-0-10 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:24:22.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-0-10 td/test-erasure-eio/COPY 2026-02-21T11:24:22.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:24:22.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:24:22.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-21T11:24:22.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74328-0-10 td/test-erasure-eio 1 10 set 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-0-10 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-21T11:24:22.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-0-10 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-0-10 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-0-10 2026-02-21T11:24:22.660 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:24:22.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-21T11:24:22.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:24:23.179 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:24:23.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:24:23.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-21T11:24:23.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-21T11:24:23.191 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:24:23.191 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:24:23.191 INFO:tasks.workunit.client.0.trial166.stderr:10 bytes copied, 2.9565e-05 s, 338 kB/s 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:23.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:23.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:23.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:23.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:24:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:24:23.502 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:23.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:23.784 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:23.785 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:24:23.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:24:23.786 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:24:23.786 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:23.786 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:24:23.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:24:23.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:23.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:23.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:23.808 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:23.802+0000 7fa893751a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:23.809 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:23.802+0000 7fa893751a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:23.810 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:23.806+0000 7fa893751a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:24.000 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:24.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:24.140 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:24.134+0000 7fa893751a40 -1 Falling back to public interface 2026-02-21T11:24:24.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:24.774 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:24.766+0000 7fa893751a40 -1 osd.1 43 log_to_monitors true 2026-02-21T11:24:25.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:25.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:25.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:25.216 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:25.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:25.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [13,43) [v2:127.0.0.1:6810/3165097767,v1:127.0.0.1:6811/3165097767] [v2:127.0.0.1:6812/3165097767,v1:127.0.0.1:6813/3165097767] exists,up f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:25.434 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:25.435 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:25.435 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:25.435 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:25.435 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:25.435 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:25.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:25.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:25.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:25.750 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:25.750 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:25.750 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:25.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:25.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:25.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:25.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803788 2026-02-21T11:24:25.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803788 2026-02-21T11:24:25.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788' 2026-02-21T11:24:25.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:25.832 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:25.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397507 2026-02-21T11:24:25.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397507 2026-02-21T11:24:25.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-210453397507' 2026-02-21T11:24:25.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:25.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:25.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411340 2026-02-21T11:24:25.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411340 2026-02-21T11:24:25.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-77309411340' 2026-02-21T11:24:25.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:25.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:26.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724550 2026-02-21T11:24:26.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724550 2026-02-21T11:24:26.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-77309411340 3-167503724550' 2026-02-21T11:24:26.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:26.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803788 2026-02-21T11:24:26.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:26.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:26.073 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803788 2026-02-21T11:24:26.073 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:26.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803788 2026-02-21T11:24:26.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-21T11:24:26.075 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803788 2026-02-21T11:24:26.075 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:26.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-21T11:24:26.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:27.289 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:27.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-21T11:24:27.503 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:28.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:24:28.505 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:28.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-21T11:24:28.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:28.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-210453397507 2026-02-21T11:24:28.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:28.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:28.728 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-210453397507 2026-02-21T11:24:28.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:28.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397507 2026-02-21T11:24:28.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-21T11:24:28.730 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 210453397507 2026-02-21T11:24:28.730 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:28.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-21T11:24:28.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:28.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411340 2026-02-21T11:24:28.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:28.957 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:28.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411340 2026-02-21T11:24:28.958 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:28.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411340 2026-02-21T11:24:28.959 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411340' 2026-02-21T11:24:28.960 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411340 2026-02-21T11:24:28.960 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:29.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411340 -lt 77309411340 2026-02-21T11:24:29.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:29.173 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-167503724550 2026-02-21T11:24:29.173 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:29.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:29.175 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-167503724550 2026-02-21T11:24:29.175 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:29.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724550 2026-02-21T11:24:29.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 167503724550' 2026-02-21T11:24:29.177 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 167503724550 2026-02-21T11:24:29.177 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:29.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-21T11:24:29.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:29.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:29.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:29.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:29.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:29.895 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:29.895 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:29.896 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:30.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:30.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:30.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:30.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:24:30.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:24:30.496 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:24:30.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-0-10 fail 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-0-10 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-21T11:24:30.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-74328-0-10 td/test-erasure-eio/COPY 2026-02-21T11:24:30.535 INFO:tasks.workunit.client.0.trial166.stderr:error getting pool-jerasure/obj-size-74328-0-10: (5) Input/output error 2026-02-21T11:24:30.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-21T11:24:30.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-21T11:24:30.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74328-0-0 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74328-0-0 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74328-0-0 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:24:30.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74328-0-0 td/test-erasure-eio/ORIGINAL 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74328-0-0 td/test-erasure-eio 0 0 set 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-0-0 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-0-0 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:24:30.559 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-0-0 2026-02-21T11:24:30.560 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-0-0 2026-02-21T11:24:30.560 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:24:30.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-21T11:24:30.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:24:31.069 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:24:31.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:24:31.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-21T11:24:31.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:31.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-21T11:24:31.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:31.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:31.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:31.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:31.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:31.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:31.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:31.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:31.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:31.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-21T11:24:31.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:31.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:31.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:31.695 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:31.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:31.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T11:24:31.697 INFO:tasks.workunit.client.0.trial166.stderr:start osd.3 2026-02-21T11:24:31.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:31.698 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-21T11:24:31.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T11:24:31.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:31.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:31.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:31.717 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:31.710+0000 7f9dc46b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:31.719 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:31.714+0000 7f9dc46b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:31.720 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:31.714+0000 7f9dc46b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:31.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:32.100 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:32.094+0000 7f9dc46b3a40 -1 Falling back to public interface 2026-02-21T11:24:32.124 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:32.749 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:32.742+0000 7f9dc46b3a40 -1 osd.3 55 log_to_monitors true 2026-02-21T11:24:33.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:33.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:33.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:33.126 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:33.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:33.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:33.343 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [39,55) [v2:127.0.0.1:6826/303526423,v1:127.0.0.1:6827/303526423] [v2:127.0.0.1:6828/303526423,v1:127.0.0.1:6829/303526423] exists,up 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:33.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:33.343 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:33.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:33.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:24:33.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:33.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:33.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:33.344 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:33.345 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:33.345 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:33.345 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:33.345 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:33.345 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:33.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:33.442 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:33.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:33.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:33.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:33.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:33.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:33.655 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:33.655 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:33.655 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:33.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:33.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:33.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:33.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803792 2026-02-21T11:24:33.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803792 2026-02-21T11:24:33.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792' 2026-02-21T11:24:33.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:33.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:33.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397510 2026-02-21T11:24:33.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397510 2026-02-21T11:24:33.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-210453397510' 2026-02-21T11:24:33.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:33.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:33.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411343 2026-02-21T11:24:33.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411343 2026-02-21T11:24:33.899 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-210453397510 2-77309411343' 2026-02-21T11:24:33.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:33.900 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:33.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=253403070467 2026-02-21T11:24:33.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 253403070467 2026-02-21T11:24:33.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-210453397510 2-77309411343 3-253403070467' 2026-02-21T11:24:33.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:33.976 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803792 2026-02-21T11:24:33.976 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:33.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:33.978 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803792 2026-02-21T11:24:33.978 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:33.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803792 2026-02-21T11:24:33.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-21T11:24:33.979 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803792 2026-02-21T11:24:33.980 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:34.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-21T11:24:34.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:34.190 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-210453397510 2026-02-21T11:24:34.190 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:34.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:34.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-210453397510 2026-02-21T11:24:34.193 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:34.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397510 2026-02-21T11:24:34.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-21T11:24:34.194 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 210453397510 2026-02-21T11:24:34.194 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:34.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397508 -lt 210453397510 2026-02-21T11:24:34.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:35.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:35.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:35.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397508 -lt 210453397510 2026-02-21T11:24:35.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:36.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:24:36.629 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:36.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397511 -lt 210453397510 2026-02-21T11:24:36.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:36.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411343 2026-02-21T11:24:36.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:36.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:36.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411343 2026-02-21T11:24:36.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:36.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411343 2026-02-21T11:24:36.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411343' 2026-02-21T11:24:36.847 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411343 2026-02-21T11:24:36.847 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:37.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411343 -lt 77309411343 2026-02-21T11:24:37.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:37.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-253403070467 2026-02-21T11:24:37.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:37.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:37.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-253403070467 2026-02-21T11:24:37.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:37.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=253403070467 2026-02-21T11:24:37.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 253403070467' 2026-02-21T11:24:37.071 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 253403070467 2026-02-21T11:24:37.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:37.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-21T11:24:37.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:37.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:37.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:37.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:37.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:37.559 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:37.559 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:37.560 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:37.560 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:37.560 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:37.560 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:37.769 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:37.769 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:37.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:37.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:38.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:38.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:38.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:38.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:24:38.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:24:38.369 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-0-0 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-0-0 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:24:38.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-0-0 td/test-erasure-eio/COPY 2026-02-21T11:24:38.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:24:38.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:24:38.413 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74328-0-0 td/test-erasure-eio 1 0 set 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-0-0 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-21T11:24:38.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-0-0 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-0-0 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-0-0 2026-02-21T11:24:38.416 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:24:38.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:24:38.626 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:38.626 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:24:38.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:24:38.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:24:38.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:24:38.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-21T11:24:38.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:24:38.974 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:24:38.985 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:24:38.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-21T11:24:38.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-21T11:24:38.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:38.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:24:38.988 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:38.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:24:38.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:38.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:38.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:38.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:38.989 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:24:39.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:39.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:24:39.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:39.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:24:39.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:39.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:24:39.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:24:39.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:39.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:39.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:39.604 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:39.598+0000 7f23699baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:39.606 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:39.598+0000 7f23699baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:39.606 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:39.602+0000 7f23699baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:39.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:24:39.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:39.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:39.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:40.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:40.028 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:40.022+0000 7f23699baa40 -1 Falling back to public interface 2026-02-21T11:24:40.553 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:40.546+0000 7f23699baa40 -1 osd.1 62 log_to_monitors true 2026-02-21T11:24:41.010 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:41.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:41.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:41.011 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:41.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:41.011 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:24:41.222 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [49,62) [v2:127.0.0.1:6810/968853155,v1:127.0.0.1:6811/968853155] [v2:127.0.0.1:6812/968853155,v1:127.0.0.1:6813/968853155] exists,up f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:24:41.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:41.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:41.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:41.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:24:41.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:41.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:41.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:41.223 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:41.224 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:41.224 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:41.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:41.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:41.224 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:41.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:41.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:41.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:41.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:41.321 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:41.321 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:41.537 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:41.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803795 2026-02-21T11:24:41.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803795 2026-02-21T11:24:41.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795' 2026-02-21T11:24:41.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:41.617 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:41.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776131 2026-02-21T11:24:41.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776131 2026-02-21T11:24:41.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-292057776131' 2026-02-21T11:24:41.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:41.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:41.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411347 2026-02-21T11:24:41.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411347 2026-02-21T11:24:41.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-292057776131 2-77309411347' 2026-02-21T11:24:41.774 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:41.774 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:41.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=253403070470 2026-02-21T11:24:41.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 253403070470 2026-02-21T11:24:41.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-292057776131 2-77309411347 3-253403070470' 2026-02-21T11:24:41.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:41.853 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803795 2026-02-21T11:24:41.854 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:41.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:41.855 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803795 2026-02-21T11:24:41.855 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:41.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803795 2026-02-21T11:24:41.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-21T11:24:41.857 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803795 2026-02-21T11:24:41.857 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:42.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-21T11:24:42.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:42.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-292057776131 2026-02-21T11:24:42.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:42.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:42.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-292057776131 2026-02-21T11:24:42.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:42.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776131 2026-02-21T11:24:42.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-21T11:24:42.070 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 292057776131 2026-02-21T11:24:42.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:42.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-21T11:24:42.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:42.300 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411347 2026-02-21T11:24:42.300 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:42.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:42.302 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411347 2026-02-21T11:24:42.302 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:42.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411347 2026-02-21T11:24:42.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411347' 2026-02-21T11:24:42.303 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411347 2026-02-21T11:24:42.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:42.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411347 -lt 77309411347 2026-02-21T11:24:42.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:42.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-253403070470 2026-02-21T11:24:42.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:42.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:42.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-253403070470 2026-02-21T11:24:42.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:42.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=253403070470 2026-02-21T11:24:42.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 253403070470' 2026-02-21T11:24:42.519 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 253403070470 2026-02-21T11:24:42.519 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:42.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070468 -lt 253403070470 2026-02-21T11:24:42.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:43.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:43.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:43.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070471 -lt 253403070470 2026-02-21T11:24:43.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:43.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:43.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:44.221 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:44.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:44.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:44.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:44.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:44.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:44.222 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:44.223 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:44.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:44.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:44.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:44.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:44.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:44.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:44.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:44.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:24:44.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:24:45.028 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:24:45.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-21T11:24:45.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-0-0 fail 2026-02-21T11:24:45.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:24:45.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:24:45.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-0-0 2026-02-21T11:24:45.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-21T11:24:45.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-21T11:24:45.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-74328-0-0 td/test-erasure-eio/COPY 2026-02-21T11:24:45.056 INFO:tasks.workunit.client.0.trial166.stderr:error getting pool-jerasure/obj-size-74328-0-0: (5) Input/output error 2026-02-21T11:24:45.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-21T11:24:45.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-21T11:24:45.061 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74328-0-256 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74328-0-256 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74328-0-256 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:24:45.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74328-0-256 td/test-erasure-eio/ORIGINAL 2026-02-21T11:24:45.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74328-0-256 td/test-erasure-eio 0 256 add 2026-02-21T11:24:45.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-0-256 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-0-256 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-0-256 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-0-256 2026-02-21T11:24:45.087 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:24:45.306 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-21T11:24:45.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:24:45.596 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:45.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:45.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:45.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:46.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:start osd.3 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:46.085 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-21T11:24:46.086 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T11:24:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:46.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:46.103 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:46.098+0000 7f29eb275a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:46.104 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:46.098+0000 7f29eb275a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:46.105 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:46.098+0000 7f29eb275a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:46.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:46.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:46.488 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:46.482+0000 7f29eb275a40 -1 Falling back to public interface 2026-02-21T11:24:46.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:47.070 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:47.062+0000 7f29eb275a40 -1 osd.3 71 log_to_monitors true 2026-02-21T11:24:47.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:47.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:47.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:47.518 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:47.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:47.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,71) [v2:127.0.0.1:6826/2510303283,v1:127.0.0.1:6827/2510303283] [v2:127.0.0.1:6828/2510303283,v1:127.0.0.1:6829/2510303283] exists,up 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:47.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:47.846 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:47.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:47.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:47.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:47.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:47.847 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:48.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:48.071 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:48.071 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:48.071 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:48.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:48.071 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:48.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:48.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803799 2026-02-21T11:24:48.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803799 2026-02-21T11:24:48.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799' 2026-02-21T11:24:48.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:48.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776134 2026-02-21T11:24:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776134 2026-02-21T11:24:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-292057776134' 2026-02-21T11:24:48.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:48.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:48.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411350 2026-02-21T11:24:48.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411350 2026-02-21T11:24:48.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-292057776134 2-77309411350' 2026-02-21T11:24:48.307 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:48.307 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:48.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=330712481795 2026-02-21T11:24:48.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 330712481795 2026-02-21T11:24:48.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-292057776134 2-77309411350 3-330712481795' 2026-02-21T11:24:48.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:48.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803799 2026-02-21T11:24:48.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:48.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:48.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803799 2026-02-21T11:24:48.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:48.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803799 2026-02-21T11:24:48.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803799' 2026-02-21T11:24:48.392 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803799 2026-02-21T11:24:48.392 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:48.599 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803796 -lt 25769803799 2026-02-21T11:24:48.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:49.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:49.601 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:49.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803796 -lt 25769803799 2026-02-21T11:24:49.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:50.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:24:50.814 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:51.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803799 -lt 25769803799 2026-02-21T11:24:51.029 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:51.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-292057776134 2026-02-21T11:24:51.030 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:51.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:51.031 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-292057776134 2026-02-21T11:24:51.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:51.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776134 2026-02-21T11:24:51.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 292057776134' 2026-02-21T11:24:51.033 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 292057776134 2026-02-21T11:24:51.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:51.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-21T11:24:51.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:51.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411350 2026-02-21T11:24:51.262 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:51.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:51.264 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411350 2026-02-21T11:24:51.264 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:51.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411350 2026-02-21T11:24:51.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411350' 2026-02-21T11:24:51.266 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411350 2026-02-21T11:24:51.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:51.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411350 -lt 77309411350 2026-02-21T11:24:51.483 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:51.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-330712481795 2026-02-21T11:24:51.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:51.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:51.485 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-330712481795 2026-02-21T11:24:51.486 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:51.487 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=330712481795 2026-02-21T11:24:51.487 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 330712481795' 2026-02-21T11:24:51.487 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 330712481795 2026-02-21T11:24:51.487 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:51.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-21T11:24:51.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:51.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:51.709 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:51.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:51.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:51.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:51.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:51.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:51.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:51.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:51.999 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:52.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:52.209 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:52.210 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:52.210 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:52.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:52.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:52.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:52.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-21T11:24:52.493 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:24:52.493 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:24:52.493 INFO:tasks.workunit.client.0.trial166.stderr:256 bytes copied, 2.5668e-05 s, 10.0 MB/s 2026-02-21T11:24:52.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:52.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:52.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:52.494 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:52.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:52.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:52.801 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:52.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:52.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-21T11:24:52.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:52.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:52.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:53.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-21T11:24:53.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:53.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:53.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T11:24:53.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:53.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:24:53.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:53.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:53.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:53.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:53.116 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:53.117 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:start osd.3 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:53.118 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:53.128 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:53.122+0000 7f00234cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:53.129 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:53.122+0000 7f00234cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:53.130 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:53.122+0000 7f00234cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:53.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:53.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:24:53.552 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:53.546+0000 7f00234cda40 -1 Falling back to public interface 2026-02-21T11:24:54.184 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:54.178+0000 7f00234cda40 -1 osd.3 78 log_to_monitors true 2026-02-21T11:24:54.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:24:54.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:54.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:24:54.543 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:54.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:54.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:24:54.759 INFO:tasks.workunit.client.0.trial166.stderr:osd.3 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [77,78) [v2:127.0.0.1:6826/2890014694,v1:127.0.0.1:6827/2890014694] [v2:127.0.0.1:6828/2890014694,v1:127.0.0.1:6829/2890014694] exists,up 6a69993f-bdb3-4e82-9c4a-b67aea3d305a 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:24:54.760 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:24:54.761 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:24:54.761 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:24:54.761 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:24:54.762 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:24:54.762 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:24:54.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:24:54.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:24:54.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:24:54.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:24:54.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:24:54.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:55.094 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:24:55.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803802 2026-02-21T11:24:55.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803802 2026-02-21T11:24:55.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802' 2026-02-21T11:24:55.171 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:55.171 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:24:55.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776137 2026-02-21T11:24:55.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776137 2026-02-21T11:24:55.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-292057776137' 2026-02-21T11:24:55.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:55.253 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:24:55.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411353 2026-02-21T11:24:55.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411353 2026-02-21T11:24:55.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-292057776137 2-77309411353' 2026-02-21T11:24:55.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:24:55.334 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:24:55.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=352187318275 2026-02-21T11:24:55.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 352187318275 2026-02-21T11:24:55.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-292057776137 2-77309411353 3-352187318275' 2026-02-21T11:24:55.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:55.410 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803802 2026-02-21T11:24:55.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:55.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:24:55.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803802 2026-02-21T11:24:55.413 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:55.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803802 2026-02-21T11:24:55.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-21T11:24:55.414 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803802 2026-02-21T11:24:55.415 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:55.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803800 -lt 25769803802 2026-02-21T11:24:55.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:24:56.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:24:56.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:24:56.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-21T11:24:56.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:56.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-292057776137 2026-02-21T11:24:56.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:56.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:24:56.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-292057776137 2026-02-21T11:24:56.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:56.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776137 2026-02-21T11:24:56.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 292057776137' 2026-02-21T11:24:56.847 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 292057776137 2026-02-21T11:24:56.847 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:24:57.062 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776138 -lt 292057776137 2026-02-21T11:24:57.063 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:57.063 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411353 2026-02-21T11:24:57.064 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:57.064 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:24:57.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411353 2026-02-21T11:24:57.065 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:57.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411353 2026-02-21T11:24:57.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411353' 2026-02-21T11:24:57.067 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411353 2026-02-21T11:24:57.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:24:57.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411353 -lt 77309411353 2026-02-21T11:24:57.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:24:57.276 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-352187318275 2026-02-21T11:24:57.276 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:24:57.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:24:57.278 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-352187318275 2026-02-21T11:24:57.279 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:24:57.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=352187318275 2026-02-21T11:24:57.280 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 352187318275' 2026-02-21T11:24:57.280 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 352187318275 2026-02-21T11:24:57.280 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:24:57.501 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 352187318275 -lt 352187318275 2026-02-21T11:24:57.501 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:24:57.502 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:57.502 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:24:57.788 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:24:58.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:24:58.005 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:24:58.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:24:58.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:24:58.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:24:58.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:24:58.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:24:58.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:24:58.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:24:58.606 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:24:58.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-0-256 2026-02-21T11:24:58.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:24:58.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:24:58.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-0-256 2026-02-21T11:24:58.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:24:58.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:24:58.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-0-256 td/test-erasure-eio/COPY 2026-02-21T11:24:58.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:24:58.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:24:58.641 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-21T11:24:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-21T11:24:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74328-0-256 td/test-erasure-eio 1 256 add 2026-02-21T11:24:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-0-256 2026-02-21T11:24:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:24:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:24:58.643 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-0-256 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:24:58.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-0-256 2026-02-21T11:24:58.645 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-0-256 2026-02-21T11:24:58.645 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:24:58.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:24:58.851 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:24:58.852 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:24:58.852 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:24:58.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:24:58.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:24:58.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-21T11:24:58.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:24:59.168 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:24:59.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:24:59.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:24:59.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:24:59.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:24:59.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:24:59.183 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:24:59.493 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:24:59.661 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:24:59.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:24:59.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:24:59.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:24:59.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:24:59.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:24:59.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:24:59.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:24:59.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:24:59.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:24:59.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:24:59.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:24:59.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:24:59.664 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:24:59.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:24:59.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:24:59.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:24:59.676 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:59.670+0000 7f49028a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:59.677 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:59.670+0000 7f49028a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:59.678 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:24:59.674+0000 7f49028a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:24:59.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:24:59.880 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:24:59.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:24:59.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:00.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:00.112 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:00.106+0000 7f49028a0a40 -1 Falling back to public interface 2026-02-21T11:25:00.735 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:00.726+0000 7f49028a0a40 -1 osd.1 85 log_to_monitors true 2026-02-21T11:25:01.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:01.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:01.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:01.091 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:01.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:01.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 91 up_thru 91 down_at 88 last_clean_interval [68,85) [v2:127.0.0.1:6810/1582569594,v1:127.0.0.1:6811/1582569594] [v2:127.0.0.1:6812/1582569594,v1:127.0.0.1:6813/1582569594] exists,up f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:25:01.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:25:01.317 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:25:01.317 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:25:01.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:25:01.317 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:25:01.318 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:25:01.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:25:01.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:25:01.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:25:01.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:25:01.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:25:01.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:25:01.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:25:01.642 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:01.642 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:25:01.642 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:25:01.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:25:01.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:01.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:25:01.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803805 2026-02-21T11:25:01.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803805 2026-02-21T11:25:01.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805' 2026-02-21T11:25:01.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:01.723 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:25:01.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=390842023939 2026-02-21T11:25:01.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 390842023939 2026-02-21T11:25:01.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-390842023939' 2026-02-21T11:25:01.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:01.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:25:01.881 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411357 2026-02-21T11:25:01.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411357 2026-02-21T11:25:01.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-390842023939 2-77309411357' 2026-02-21T11:25:01.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:01.882 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:25:01.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=352187318278 2026-02-21T11:25:01.960 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 352187318278 2026-02-21T11:25:01.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-390842023939 2-77309411357 3-352187318278' 2026-02-21T11:25:01.961 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:01.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803805 2026-02-21T11:25:01.961 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:01.962 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:25:01.963 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803805 2026-02-21T11:25:01.963 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:01.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803805 2026-02-21T11:25:01.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803805' 2026-02-21T11:25:01.964 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803805 2026-02-21T11:25:01.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:02.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803805 -lt 25769803805 2026-02-21T11:25:02.182 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:02.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-390842023939 2026-02-21T11:25:02.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:02.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:25:02.184 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-390842023939 2026-02-21T11:25:02.185 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:02.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=390842023939 2026-02-21T11:25:02.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 390842023939' 2026-02-21T11:25:02.186 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 390842023939 2026-02-21T11:25:02.187 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:02.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 390842023939 2026-02-21T11:25:02.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:03.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:25:03.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:03.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 390842023939 2026-02-21T11:25:03.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:04.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:25:04.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:04.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-21T11:25:04.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:04.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411357 2026-02-21T11:25:04.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:04.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:25:04.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411357 2026-02-21T11:25:04.846 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:04.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411357 2026-02-21T11:25:04.847 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411357' 2026-02-21T11:25:04.847 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411357 2026-02-21T11:25:04.847 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:25:05.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411357 -lt 77309411357 2026-02-21T11:25:05.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:05.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-352187318278 2026-02-21T11:25:05.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:05.070 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:25:05.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-352187318278 2026-02-21T11:25:05.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:05.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=352187318278 2026-02-21T11:25:05.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 352187318278' 2026-02-21T11:25:05.072 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 352187318278 2026-02-21T11:25:05.073 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:05.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 352187318278 -lt 352187318278 2026-02-21T11:25:05.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:25:05.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:05.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:05.572 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:25:05.573 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:05.573 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:05.573 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:05.573 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:05.573 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:05.574 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:05.574 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:05.780 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:25:05.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:05.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:05.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:25:06.066 INFO:tasks.workunit.client.0.trial166.stderr:256 bytes copied, 2.4236e-05 s, 10.6 MB/s 2026-02-21T11:25:06.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:06.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:25:06.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:25:06.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:25:06.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:25:06.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:06.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:06.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:06.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:06.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:25:06.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:25:06.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:25:06.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:25:06.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:06.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:06.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:25:06.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:06.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:25:06.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:06.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:06.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:06.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:25:06.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:25:06.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:25:06.720 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:06.714+0000 7fd0479e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:06.722 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:06.714+0000 7fd0479e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:06.723 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:06.718+0000 7fd0479e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:25:06.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:06.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:07.064 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:07.058+0000 7fd0479e9a40 -1 Falling back to public interface 2026-02-21T11:25:07.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:07.672 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:07.666+0000 7fd0479e9a40 -1 osd.1 92 log_to_monitors true 2026-02-21T11:25:08.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:08.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:08.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:08.133 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:08.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:08.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 96 up_thru 96 down_at 93 last_clean_interval [91,92) [v2:127.0.0.1:6810/1409648301,v1:127.0.0.1:6811/1409648301] [v2:127.0.0.1:6812/1409648301,v1:127.0.0.1:6813/1409648301] exists,up f737fd3b-40c0-46cf-b0f1-4873b6311b7c 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:25:08.347 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:25:08.348 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:25:08.348 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:25:08.348 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:25:08.348 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:25:08.348 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:25:08.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:25:08.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:25:08.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:25:08.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:25:08.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:25:08.457 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:08.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:25:08.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803809 2026-02-21T11:25:08.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803809 2026-02-21T11:25:08.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809' 2026-02-21T11:25:08.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:08.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:25:08.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=412316860419 2026-02-21T11:25:08.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 412316860419 2026-02-21T11:25:08.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-412316860419' 2026-02-21T11:25:08.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:08.832 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:25:08.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411360 2026-02-21T11:25:08.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411360 2026-02-21T11:25:08.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-412316860419 2-77309411360' 2026-02-21T11:25:08.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:08.911 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:25:08.990 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=352187318281 2026-02-21T11:25:08.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 352187318281 2026-02-21T11:25:08.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-412316860419 2-77309411360 3-352187318281' 2026-02-21T11:25:08.991 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:08.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803809 2026-02-21T11:25:08.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:08.992 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:25:08.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803809 2026-02-21T11:25:08.993 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:08.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803809 2026-02-21T11:25:08.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803809' 2026-02-21T11:25:08.995 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803809 2026-02-21T11:25:08.995 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:09.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803809 2026-02-21T11:25:09.203 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:10.204 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:25:10.204 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:10.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803809 -lt 25769803809 2026-02-21T11:25:10.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:10.428 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-412316860419 2026-02-21T11:25:10.428 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:10.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:25:10.430 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-412316860419 2026-02-21T11:25:10.430 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:10.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=412316860419 2026-02-21T11:25:10.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 412316860419' 2026-02-21T11:25:10.431 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 412316860419 2026-02-21T11:25:10.432 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:10.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-21T11:25:10.655 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411360 2026-02-21T11:25:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:10.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:25:10.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411360 2026-02-21T11:25:10.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:10.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411360 2026-02-21T11:25:10.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411360' 2026-02-21T11:25:10.659 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 77309411360 2026-02-21T11:25:10.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:25:10.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411360 -lt 77309411360 2026-02-21T11:25:10.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:10.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-352187318281 2026-02-21T11:25:10.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:10.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:25:10.872 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-352187318281 2026-02-21T11:25:10.872 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:10.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=352187318281 2026-02-21T11:25:10.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 352187318281' 2026-02-21T11:25:10.874 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 352187318281 2026-02-21T11:25:10.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:11.088 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 352187318282 -lt 352187318281 2026-02-21T11:25:11.089 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:25:11.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:11.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:11.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:25:11.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:11.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:11.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:11.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:11.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:11.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:11.378 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:11.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:25:11.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:11.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:11.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:11.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:25:11.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:25:11.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:25:11.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:25:11.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:25:12.199 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-0-256 fail 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-0-256 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-21T11:25:12.209 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-74328-0-256 td/test-erasure-eio/COPY 2026-02-21T11:25:12.228 INFO:tasks.workunit.client.0.trial166.stderr:error getting pool-jerasure/obj-size-74328-0-256: (5) Input/output error 2026-02-21T11:25:12.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-21T11:25:12.230 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:25:12.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:25:12.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:25:12.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:25:12.452 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:25:12.462 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:25:12.760 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:12.770 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:12.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:12.882 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:25:12.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:25:12.883 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:25:12.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:25:12.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:25:12.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:25:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:25:12.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:25:12.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:25:12.887 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:25:12.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:25:12.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:25:12.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:25:12.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:25:12.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:12.914 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:12.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:25:12.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:25:12.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:25:12.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:25:12.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:12.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:12.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:12.918 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:25:12.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:25:12.920 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:25:12.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:25:12.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:25:12.921 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:25:12.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:25:12.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:25:12.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:25:12.924 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:25:12.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:25:12.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:25:12.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:25:12.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:25:12.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:12.928 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:12.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:25:12.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:25:12.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:25:12.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:25:12.930 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:25:12.931 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:12.931 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:12.931 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:25:12.932 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:25:12.933 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:25:12.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:25:12.934 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:12.956 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:25:12.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:25:12.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:25:12.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:25:12.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:25:12.987 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:12.988 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:12.989 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:25:12.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:25:12.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:25:13.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:25:13.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:25:13.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:25:13.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:25:13.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:13.045 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:25:13.046 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:25:13.046 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:25:13.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:13.212 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:13.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:25:13.214 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:25:13.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:25:13.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:25:13.342 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:25:13.350 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:25:14.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:25:14.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:25:14.402 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:25:14.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:25:14.409 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:25:12.970+0000 7f7aebc7bec0 0 load: jerasure load: lrc 2026-02-21T11:25:14.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-21T11:25:14.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-21T11:25:14.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-21T11:25:14.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:25:14.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:25:14.410 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:25:14.411 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:25:14.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:14.413 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:25:14.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:25:14.415 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:25:14.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d4143add-357b-4dc3-a157-e30c27b3248a 2026-02-21T11:25:14.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d4143add-357b-4dc3-a157-e30c27b3248a' 2026-02-21T11:25:14.416 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 d4143add-357b-4dc3-a157-e30c27b3248a 2026-02-21T11:25:14.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:25:14.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAalplpCAJpGRAArtfNUns98pNjUeRkbY450g== 2026-02-21T11:25:14.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAalplpCAJpGRAArtfNUns98pNjUeRkbY450g=="}' 2026-02-21T11:25:14.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d4143add-357b-4dc3-a157-e30c27b3248a -i td/test-erasure-eio/0/new.json 2026-02-21T11:25:14.537 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:25:14.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:25:14.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAalplpCAJpGRAArtfNUns98pNjUeRkbY450g== --osd-uuid d4143add-357b-4dc3-a157-e30c27b3248a 2026-02-21T11:25:14.561 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:14.554+0000 7fc5939eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:14.562 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:14.558+0000 7fc5939eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:14.563 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:14.558+0000 7fc5939eca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:14.564 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:14.558+0000 7fc5939eca40 -1 bdev(0x55bf3b800400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:25:14.564 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:14.558+0000 7fc5939eca40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:25:15.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:25:15.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:25:15.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:25:15.254 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:25:15.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:25:15.366 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:25:15.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:25:15.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:15.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:25:15.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:25:15.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:25:15.386 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:15.378+0000 7f21542d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:15.388 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:15.382+0000 7f21542d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:15.390 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:15.382+0000 7f21542d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:15.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:25:15.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:15.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:25:15.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:15.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:15.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:15.505 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:15.506 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:25:15.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:15.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:25:15.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:15.828 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:15.822+0000 7f21542d6a40 -1 Falling back to public interface 2026-02-21T11:25:16.449 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:16.442+0000 7f21542d6a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:25:16.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:16.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:16.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:16.617 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:25:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:25:16.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:17.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:17.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:17.835 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:25:17.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:25:17.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:17.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:25:18.048 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1260972834,v1:127.0.0.1:6803/1260972834] [v2:127.0.0.1:6804/1260972834,v1:127.0.0.1:6805/1260972834] exists,up d4143add-357b-4dc3-a157-e30c27b3248a 2026-02-21T11:25:18.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:18.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:25:18.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:25:18.055 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:18.056 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:18.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:25:18.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:25:18.059 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:25:18.060 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d387fe64-1512-4dcd-a262-240489a1b19a' 2026-02-21T11:25:18.060 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:25:18.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAelplpmLLTAxAA1CvgGiTB2tieSD/k85oazQ== 2026-02-21T11:25:18.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAelplpmLLTAxAA1CvgGiTB2tieSD/k85oazQ=="}' 2026-02-21T11:25:18.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d387fe64-1512-4dcd-a262-240489a1b19a -i td/test-erasure-eio/1/new.json 2026-02-21T11:25:18.285 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:25:18.296 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:25:18.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAelplpmLLTAxAA1CvgGiTB2tieSD/k85oazQ== --osd-uuid d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:25:18.320 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:18.314+0000 7f5a92284a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:18.322 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:18.314+0000 7f5a92284a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:18.323 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:18.318+0000 7f5a92284a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:18.323 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:18.318+0000 7f5a92284a40 -1 bdev(0x55e117366400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:25:18.324 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:18.318+0000 7f5a92284a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:25:18.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:25:18.954 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:25:18.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:25:18.955 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:25:18.955 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:25:19.251 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:25:19.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:25:19.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:19.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:25:19.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:25:19.251 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:25:19.274 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:19.265+0000 7fd4b4deba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:19.276 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:19.269+0000 7fd4b4deba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:19.277 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:19.269+0000 7fd4b4deba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:19.464 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:25:19.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:19.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:19.465 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:19.632 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:19.625+0000 7fd4b4deba40 -1 Falling back to public interface 2026-02-21T11:25:19.682 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:20.291 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:20.285+0000 7fd4b4deba40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:25:20.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:20.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:20.684 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:25:20.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:20.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:20.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:20.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:21.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:21.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:21.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:25:21.898 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:25:21.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:21.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:22.108 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3468299912,v1:127.0.0.1:6811/3468299912] [v2:127.0.0.1:6812/3468299912,v1:127.0.0.1:6813/3468299912] exists,up d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:25:22.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:22.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:22.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:22.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:25:22.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:22.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:25:22.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:25:22.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:25:22.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:25:22.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f438ce0d-4634-48c2-9a9f-68146e3924d4' 2026-02-21T11:25:22.115 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:25:22.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:25:22.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAilplpCbFZBxAAi1rfLM4bMOxO5U+uKhbRRQ== 2026-02-21T11:25:22.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAilplpCbFZBxAAi1rfLM4bMOxO5U+uKhbRRQ=="}' 2026-02-21T11:25:22.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f438ce0d-4634-48c2-9a9f-68146e3924d4 -i td/test-erasure-eio/2/new.json 2026-02-21T11:25:22.334 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:25:22.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:25:22.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAilplpCbFZBxAAi1rfLM4bMOxO5U+uKhbRRQ== --osd-uuid f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:25:22.363 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:22.357+0000 7f1e4ba0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:22.364 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:22.357+0000 7f1e4ba0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:22.365 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:22.357+0000 7f1e4ba0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:22.366 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:22.357+0000 7f1e4ba0ca40 -1 bdev(0x55f456834400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:25:22.366 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:22.357+0000 7f1e4ba0ca40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:25:22.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:25:22.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:25:22.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:25:22.983 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:25:22.983 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:25:23.275 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:25:23.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:25:23.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:23.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:25:23.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:25:23.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:25:23.298 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:23.289+0000 7f46ec2fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:23.300 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:23.293+0000 7f46ec2fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:23.301 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:23.293+0000 7f46ec2fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:23.489 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:25:23.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:23.748 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:23.741+0000 7f46ec2fba40 -1 Falling back to public interface 2026-02-21T11:25:24.357 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:24.349+0000 7f46ec2fba40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:25:24.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:24.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:24.711 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:25:24.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:24.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:24.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:25:24.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:25.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:25.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:25.926 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:25:25.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:25:25.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:25.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:25:26.138 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3690979289,v1:127.0.0.1:6819/3690979289] [v2:127.0.0.1:6820/3690979289,v1:127.0.0.1:6821/3690979289] exists,up f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:25:26.144 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:26.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:26.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:26.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:25:26.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:25:26.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:26.145 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:25:26.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:26.147 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:25:26.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:25:26.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=31bdfdd4-e986-4abf-99d3-9b069771a25f 2026-02-21T11:25:26.150 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 31bdfdd4-e986-4abf-99d3-9b069771a25f 2026-02-21T11:25:26.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 31bdfdd4-e986-4abf-99d3-9b069771a25f' 2026-02-21T11:25:26.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:25:26.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAmlplphEkPCRAAFL0E6cNO/fn9oS7qcHL96A== 2026-02-21T11:25:26.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAmlplphEkPCRAAFL0E6cNO/fn9oS7qcHL96A=="}' 2026-02-21T11:25:26.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 31bdfdd4-e986-4abf-99d3-9b069771a25f -i td/test-erasure-eio/3/new.json 2026-02-21T11:25:26.372 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:25:26.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:25:26.384 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmlplphEkPCRAAFL0E6cNO/fn9oS7qcHL96A== --osd-uuid 31bdfdd4-e986-4abf-99d3-9b069771a25f 2026-02-21T11:25:26.408 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:26.401+0000 7fdb9b85ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:26.410 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:26.405+0000 7fdb9b85ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:26.411 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:26.405+0000 7fdb9b85ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:26.412 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:26.405+0000 7fdb9b85ca40 -1 bdev(0x560e15252400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:25:26.412 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:26.405+0000 7fdb9b85ca40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:25:26.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:25:26.994 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:25:26.995 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:25:26.995 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:25:26.996 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:25:27.286 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:25:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:25:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:25:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:25:27.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:25:27.311 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:27.309+0000 7fcee4963a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:27.312 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:27.309+0000 7fcee4963a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:27.314 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:27.309+0000 7fcee4963a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:27.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:25:27.656 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:27.653+0000 7fcee4963a40 -1 Falling back to public interface 2026-02-21T11:25:27.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:28.325 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:28.321+0000 7fcee4963a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:25:28.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:28.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:28.710 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:25:28.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:28.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:28.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:25:28.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:29.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:29.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:29.924 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:25:29.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:25:29.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:29.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:25:30.146 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/800957907,v1:127.0.0.1:6827/800957907] [v2:127.0.0.1:6828/800957907,v1:127.0.0.1:6829/800957907] exists,up 31bdfdd4-e986-4abf-99d3-9b069771a25f 2026-02-21T11:25:30.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:30.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:30.146 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:30.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:25:30.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:25:30.147 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:25:30.147 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:25:30.147 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:30.147 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:30.148 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:25:30.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:25:30.148 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:25:30.212 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:25:30.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:25:30.220 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:25:15.826+0000 7f21542d6a40 0 load: jerasure load: lrc 2026-02-21T11:25:30.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-21T11:25:30.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:25:30.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:25:30.222 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:25:30.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:25:30.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:25:30.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:25:30.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:25:30.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:25:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:25:30.504 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:25:30.876 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:25:30.886 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:25:31.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:25:31.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:25:31.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:25:31.888 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:25:31.888 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:25:31.889 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:25:31.889 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:25:31.889 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:25:31.889 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:25:31.889 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:25:32.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:25:32.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:25:32.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:25:32.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:25:32.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:25:32.001 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:25:32.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:25:32.213 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:32.213 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:25:32.213 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:25:32.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:25:32.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:32.214 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:25:32.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-21T11:25:32.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-21T11:25:32.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-21T11:25:32.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:32.293 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:25:32.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574853 2026-02-21T11:25:32.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574853 2026-02-21T11:25:32.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-21T11:25:32.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:32.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:25:32.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378628 2026-02-21T11:25:32.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378628 2026-02-21T11:25:32.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628' 2026-02-21T11:25:32.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:32.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:25:32.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116995 2026-02-21T11:25:32.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116995 2026-02-21T11:25:32.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628 3-115964116995' 2026-02-21T11:25:32.528 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:32.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-21T11:25:32.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:32.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:25:32.530 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-21T11:25:32.530 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:32.531 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-21T11:25:32.531 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803781 2026-02-21T11:25:32.532 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-21T11:25:32.532 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:32.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-21T11:25:32.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:33.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:25:33.749 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:33.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-21T11:25:33.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:33.966 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574853 2026-02-21T11:25:33.966 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:33.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:25:33.968 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574853 2026-02-21T11:25:33.968 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:33.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574853 2026-02-21T11:25:33.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-21T11:25:33.969 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574853 2026-02-21T11:25:33.970 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:34.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-21T11:25:34.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:34.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378628 2026-02-21T11:25:34.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:34.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:25:34.193 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378628 2026-02-21T11:25:34.194 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:34.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378628 2026-02-21T11:25:34.195 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378628 2026-02-21T11:25:34.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-21T11:25:34.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:25:34.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-21T11:25:34.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:34.414 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116995 2026-02-21T11:25:34.414 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:34.415 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:25:34.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116995 2026-02-21T11:25:34.416 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:34.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116995 2026-02-21T11:25:34.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-21T11:25:34.418 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 115964116995 2026-02-21T11:25:34.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:34.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-21T11:25:34.639 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:25:34.640 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:34.640 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:34.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:35.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:25:35.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:35.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:35.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:35.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:25:35.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:25:35.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:25:35.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74328-1-10 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74328-1-10 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:25:35.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74328-1-10 td/test-erasure-eio/ORIGINAL 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74328-1-10 td/test-erasure-eio 1 10 set 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-1-10 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:25:35.445 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-1-10 2026-02-21T11:25:35.446 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:35.446 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:25:35.657 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:25:35.657 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:35.657 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:25:35.657 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:25:35.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:25:35.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:25:35.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-21T11:25:35.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:25:35.973 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:25:35.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:25:35.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-21T11:25:35.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-21T11:25:35.983 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:25:35.983 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:25:35.983 INFO:tasks.workunit.client.0.trial166.stderr:10 bytes copied, 3.4645e-05 s, 289 kB/s 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:35.984 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:36.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:36.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:36.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:25:36.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:25:36.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:25:36.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:25:36.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:36.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:36.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:36.595 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:36.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:25:36.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:25:36.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:25:36.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:25:36.597 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:25:36.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:36.598 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:25:36.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:25:36.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:25:36.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:25:36.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:25:36.613 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:36.609+0000 7fb98a851a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:36.615 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:36.613+0000 7fb98a851a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:36.615 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:36.613+0000 7fb98a851a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:36.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:37.004 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:37.001+0000 7fb98a851a40 -1 Falling back to public interface 2026-02-21T11:25:37.016 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:37.590 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:37.585+0000 7fb98a851a40 -1 osd.1 36 log_to_monitors true 2026-02-21T11:25:38.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:38.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:38.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:38.018 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:38.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:38.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:38.231 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [13,36) [v2:127.0.0.1:6810/2479582788,v1:127.0.0.1:6811/2479582788] [v2:127.0.0.1:6812/2479582788,v1:127.0.0.1:6813/2479582788] exists,up d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:25:38.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:25:38.233 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:25:38.233 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:25:38.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:25:38.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:25:38.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:25:38.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:25:38.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:25:38.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:25:38.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:25:38.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:25:38.339 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:25:38.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:25:38.552 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:38.553 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:25:38.553 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:25:38.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:25:38.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:38.553 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:25:38.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803785 2026-02-21T11:25:38.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803785 2026-02-21T11:25:38.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785' 2026-02-21T11:25:38.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:38.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:25:38.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659139 2026-02-21T11:25:38.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659139 2026-02-21T11:25:38.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-176093659139' 2026-02-21T11:25:38.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:38.712 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:25:38.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378631 2026-02-21T11:25:38.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378631 2026-02-21T11:25:38.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-176093659139 2-81604378631' 2026-02-21T11:25:38.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:38.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:25:38.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116998 2026-02-21T11:25:38.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116998 2026-02-21T11:25:38.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-176093659139 2-81604378631 3-115964116998' 2026-02-21T11:25:38.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:38.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803785 2026-02-21T11:25:38.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:38.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:25:38.872 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803785 2026-02-21T11:25:38.872 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803785 2026-02-21T11:25:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-21T11:25:38.874 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803785 2026-02-21T11:25:38.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:39.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-21T11:25:39.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:40.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:25:40.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:40.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-21T11:25:40.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:40.299 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-176093659139 2026-02-21T11:25:40.299 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:40.301 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:25:40.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-176093659139 2026-02-21T11:25:40.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:40.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659139 2026-02-21T11:25:40.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 176093659139' 2026-02-21T11:25:40.303 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 176093659139 2026-02-21T11:25:40.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:40.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-21T11:25:40.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:40.526 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378631 2026-02-21T11:25:40.526 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:40.527 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:25:40.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378631 2026-02-21T11:25:40.528 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:40.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378631 2026-02-21T11:25:40.529 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378631' 2026-02-21T11:25:40.530 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 81604378631 2026-02-21T11:25:40.530 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:25:40.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378631 -lt 81604378631 2026-02-21T11:25:40.758 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:40.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116998 2026-02-21T11:25:40.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:40.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:25:40.760 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116998 2026-02-21T11:25:40.760 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:40.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116998 2026-02-21T11:25:40.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-21T11:25:40.762 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964116998 2026-02-21T11:25:40.762 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:40.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116999 -lt 115964116998 2026-02-21T11:25:40.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:25:40.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:40.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:41.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:25:41.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:41.268 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:41.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:25:41.478 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:41.479 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:41.479 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:41.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:25:41.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:25:41.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:25:41.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:25:41.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:25:42.070 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:25:42.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:42.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:25:42.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:25:42.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-1-10 2026-02-21T11:25:42.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:25:42.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:25:42.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-1-10 td/test-erasure-eio/COPY 2026-02-21T11:25:42.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:25:42.110 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:25:42.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74328-1-10 td/test-erasure-eio 2 10 set 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-1-10 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:25:42.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-1-10 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:42.115 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-21T11:25:42.328 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:25:42.643 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:25:42.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:25:42.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-21T11:25:42.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-21T11:25:42.656 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:25:42.656 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:10 bytes copied, 3.3021e-05 s, 303 kB/s 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:25:42.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:42.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:25:42.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:25:42.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-21T11:25:42.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:25:42.658 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:25:42.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:42.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:42.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:42.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:42.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-21T11:25:42.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74328-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:43.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-21T11:25:43.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:25:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:25:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:25:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:25:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:43.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:43.263 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:43.264 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:43.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr:start osd.2 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:25:43.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:25:43.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:25:43.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:25:43.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:25:43.277 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:43.273+0000 7f23481cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:43.279 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:43.277+0000 7f23481cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:43.279 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:43.277+0000 7f23481cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:43.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:43.479 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:25:43.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:43.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:25:43.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:43.708 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:43.705+0000 7f23481cea40 -1 Falling back to public interface 2026-02-21T11:25:44.351 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:44.349+0000 7f23481cea40 -1 osd.2 45 log_to_monitors true 2026-02-21T11:25:44.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:44.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:44.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:44.688 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:44.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:44.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:25:44.900 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 up in weight 1 up_from 50 up_thru 38 down_at 47 last_clean_interval [19,45) [v2:127.0.0.1:6818/3199687704,v1:127.0.0.1:6819/3199687704] [v2:127.0.0.1:6820/3199687704,v1:127.0.0.1:6821/3199687704] exists,up f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:25:44.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:25:44.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:25:44.902 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:25:44.902 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:25:44.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:25:44.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:25:44.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:25:45.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:25:45.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:25:45.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:25:45.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:25:45.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:25:45.002 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:25:45.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:25:45.217 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:45.217 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:25:45.217 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:25:45.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:25:45.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:45.218 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:25:45.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803788 2026-02-21T11:25:45.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803788 2026-02-21T11:25:45.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788' 2026-02-21T11:25:45.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:45.299 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:25:45.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659142 2026-02-21T11:25:45.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659142 2026-02-21T11:25:45.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-176093659142' 2026-02-21T11:25:45.381 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:45.381 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:25:45.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364803 2026-02-21T11:25:45.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364803 2026-02-21T11:25:45.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-176093659142 2-214748364803' 2026-02-21T11:25:45.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:45.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:25:45.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117002 2026-02-21T11:25:45.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117002 2026-02-21T11:25:45.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-176093659142 2-214748364803 3-115964117002' 2026-02-21T11:25:45.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:45.543 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803788 2026-02-21T11:25:45.543 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:45.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:25:45.544 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803788 2026-02-21T11:25:45.544 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:45.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803788 2026-02-21T11:25:45.546 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-21T11:25:45.546 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803788 2026-02-21T11:25:45.546 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:45.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-21T11:25:45.773 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:45.774 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-176093659142 2026-02-21T11:25:45.774 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:45.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:25:45.775 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-176093659142 2026-02-21T11:25:45.776 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:45.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659142 2026-02-21T11:25:45.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 176093659142' 2026-02-21T11:25:45.777 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 176093659142 2026-02-21T11:25:45.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:46.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659142 -lt 176093659142 2026-02-21T11:25:46.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:46.004 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-214748364803 2026-02-21T11:25:46.004 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:46.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:25:46.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-214748364803 2026-02-21T11:25:46.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:46.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364803 2026-02-21T11:25:46.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 214748364803' 2026-02-21T11:25:46.007 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 214748364803 2026-02-21T11:25:46.008 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:25:46.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-21T11:25:46.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:46.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117002 2026-02-21T11:25:46.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:46.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:25:46.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117002 2026-02-21T11:25:46.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:46.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117002 2026-02-21T11:25:46.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117002' 2026-02-21T11:25:46.236 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117002 2026-02-21T11:25:46.236 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:46.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117000 -lt 115964117002 2026-02-21T11:25:46.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:47.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:25:47.452 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:47.674 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117002 -lt 115964117002 2026-02-21T11:25:47.674 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:25:47.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:47.675 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:47.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:25:47.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:47.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:47.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:47.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:47.965 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:47.966 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:47.966 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:48.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=2 2026-02-21T11:25:48.180 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:48.181 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:48.181 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:48.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 = 5 2026-02-21T11:25:48.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 2 '!=' -1 2026-02-21T11:25:48.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T11:25:48.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=2 2026-02-21T11:25:48.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T11:25:48.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T11:25:48.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T11:25:48.577 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:48.577 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:48.577 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:48.577 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:48.577 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:48.578 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:48.578 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:48.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=2 2026-02-21T11:25:48.792 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:48.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:48.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 = 5 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 2 '!=' 2 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T11:25:49.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T11:25:49.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T11:25:49.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T11:25:49.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-21T11:25:49.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T11:25:49.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T11:25:49.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T11:25:49.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:49.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:49.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:49.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:49.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:49.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:49.571 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:49.781 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:25:49.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:49.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:49.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:25:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:25:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:25:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:25:50.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:25:50.323 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:25:50.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-21T11:25:50.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-1-10 2026-02-21T11:25:50.333 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:25:50.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:25:50.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-1-10 2026-02-21T11:25:50.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:25:50.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:25:50.334 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-1-10 td/test-erasure-eio/COPY 2026-02-21T11:25:50.360 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:25:50.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:25:50.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:25:50.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-21T11:25:50.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74328-1-0 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74328-1-0 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:25:50.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74328-1-0 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:25:50.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74328-1-0 td/test-erasure-eio/ORIGINAL 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74328-1-0 td/test-erasure-eio 1 0 set 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-1-0 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:25:50.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:25:50.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:25:50.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-1-0 2026-02-21T11:25:50.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:25:50.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-1-0 2026-02-21T11:25:50.391 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-1-0 2026-02-21T11:25:50.391 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:25:50.597 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-21T11:25:50.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:25:50.890 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:25:50.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:25:50.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-21T11:25:50.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:25:50.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:25:50.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:25:50.903 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:50.903 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:50.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:50.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:50.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:51.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:51.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:51.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:25:51.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:25:51.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:25:51.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:25:51.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:51.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:51.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:25:51.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:51.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:25:51.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:51.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:51.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:25:51.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:51.518 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:25:51.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:25:51.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:25:51.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:25:51.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:25:51.531 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:51.529+0000 7f7bc2e3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:51.533 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:51.529+0000 7f7bc2e3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:51.534 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:51.529+0000 7f7bc2e3ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:51.731 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:25:51.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:51.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:51.880 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:51.877+0000 7f7bc2e3ea40 -1 Falling back to public interface 2026-02-21T11:25:51.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:25:52.561 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:52.557+0000 7f7bc2e3ea40 -1 osd.1 53 log_to_monitors true 2026-02-21T11:25:52.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:25:52.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:52.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:25:52.940 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:52.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:52.940 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [41,53) [v2:127.0.0.1:6810/1118930155,v1:127.0.0.1:6811/1118930155] [v2:127.0.0.1:6812/1118930155,v1:127.0.0.1:6813/1118930155] exists,up d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:25:53.153 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:25:53.154 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:25:53.154 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:25:53.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:25:53.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:25:53.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:25:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:25:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:25:53.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:25:53.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:25:53.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:25:53.266 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:25:53.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:25:53.480 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:53.480 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:25:53.480 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:25:53.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:25:53.480 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:53.481 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:25:53.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803792 2026-02-21T11:25:53.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803792 2026-02-21T11:25:53.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792' 2026-02-21T11:25:53.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:53.560 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:25:53.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=253403070467 2026-02-21T11:25:53.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 253403070467 2026-02-21T11:25:53.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-253403070467' 2026-02-21T11:25:53.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:53.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:25:53.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364806 2026-02-21T11:25:53.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364806 2026-02-21T11:25:53.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-253403070467 2-214748364806' 2026-02-21T11:25:53.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:25:53.725 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:25:53.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117005 2026-02-21T11:25:53.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117005 2026-02-21T11:25:53.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-253403070467 2-214748364806 3-115964117005' 2026-02-21T11:25:53.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:53.804 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803792 2026-02-21T11:25:53.804 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:53.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:25:53.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803792 2026-02-21T11:25:53.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:53.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803792 2026-02-21T11:25:53.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-21T11:25:53.808 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803792 2026-02-21T11:25:53.808 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:54.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-21T11:25:54.020 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:55.021 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:25:55.022 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:55.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-21T11:25:55.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:25:56.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:25:56.239 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:25:56.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-21T11:25:56.451 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:56.452 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-253403070467 2026-02-21T11:25:56.452 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:56.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:25:56.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-253403070467 2026-02-21T11:25:56.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:56.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=253403070467 2026-02-21T11:25:56.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 253403070467' 2026-02-21T11:25:56.456 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 253403070467 2026-02-21T11:25:56.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:25:56.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-21T11:25:56.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:56.684 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-214748364806 2026-02-21T11:25:56.684 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:56.685 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:25:56.686 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-214748364806 2026-02-21T11:25:56.686 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:56.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364806 2026-02-21T11:25:56.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 214748364806' 2026-02-21T11:25:56.689 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 214748364806 2026-02-21T11:25:56.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:25:56.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364807 -lt 214748364806 2026-02-21T11:25:56.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:25:56.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117005 2026-02-21T11:25:56.911 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:25:56.912 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:25:56.912 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117005 2026-02-21T11:25:56.913 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:25:56.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117005 2026-02-21T11:25:56.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117005' 2026-02-21T11:25:56.914 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117005 2026-02-21T11:25:56.915 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:25:57.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117006 -lt 115964117005 2026-02-21T11:25:57.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:25:57.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:57.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:57.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:25:57.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:25:57.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:25:57.611 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:25:57.611 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:25:57.611 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:25:57.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:25:57.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:25:57.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:25:57.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:25:57.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:25:58.223 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-1-0 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-1-0 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:25:58.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-1-0 td/test-erasure-eio/COPY 2026-02-21T11:25:58.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:25:58.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:25:58.258 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74328-1-0 td/test-erasure-eio 2 0 set 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-1-0 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:25:58.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-1-0 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:25:58.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-1-0 2026-02-21T11:25:58.261 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-1-0 2026-02-21T11:25:58.261 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:25:58.478 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-21T11:25:58.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:25:58.785 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:25:58.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-21T11:25:58.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-21T11:25:58.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:25:58.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:25:58.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:25:58.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:25:58.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:25:58.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:25:58.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:25:59.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:25:59.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:59.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:25:59.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:25:59.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-21T11:25:59.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:25:59.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-21T11:25:59.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74328-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:25:59.397 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:25:59.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:25:59.399 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:25:59.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:start osd.2 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:25:59.402 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:25:59.417 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:59.413+0000 7f1de485fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:59.419 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:59.417+0000 7f1de485fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:59.420 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:59.417+0000 7f1de485fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:25:59.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:25:59.768 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:25:59.765+0000 7f1de485fa40 -1 Falling back to public interface 2026-02-21T11:25:59.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:00.668 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:00.665+0000 7f1de485fa40 -1 osd.2 63 log_to_monitors true 2026-02-21T11:26:00.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:00.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:00.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:00.836 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:00.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:00.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 up in weight 1 up_from 68 up_thru 56 down_at 65 last_clean_interval [50,63) [v2:127.0.0.1:6818/1882693425,v1:127.0.0.1:6819/1882693425] [v2:127.0.0.1:6820/1882693425,v1:127.0.0.1:6821/1882693425] exists,up f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:26:01.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:26:01.048 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:26:01.048 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:26:01.048 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:26:01.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:26:01.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:26:01.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:26:01.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:26:01.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:26:01.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:26:01.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:26:01.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:26:01.155 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:01.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:26:01.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803795 2026-02-21T11:26:01.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803795 2026-02-21T11:26:01.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795' 2026-02-21T11:26:01.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:01.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:26:01.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=253403070470 2026-02-21T11:26:01.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 253403070470 2026-02-21T11:26:01.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-253403070470' 2026-02-21T11:26:01.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:01.538 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:26:01.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776131 2026-02-21T11:26:01.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776131 2026-02-21T11:26:01.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-253403070470 2-292057776131' 2026-02-21T11:26:01.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:01.620 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:26:01.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117009 2026-02-21T11:26:01.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117009 2026-02-21T11:26:01.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-253403070470 2-292057776131 3-115964117009' 2026-02-21T11:26:01.700 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:01.700 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803795 2026-02-21T11:26:01.701 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:01.702 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:26:01.702 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803795 2026-02-21T11:26:01.703 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:01.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803795 2026-02-21T11:26:01.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-21T11:26:01.704 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803795 2026-02-21T11:26:01.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:01.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-21T11:26:01.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:01.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-253403070470 2026-02-21T11:26:01.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:01.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:26:01.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-253403070470 2026-02-21T11:26:01.927 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:01.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=253403070470 2026-02-21T11:26:01.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 253403070470' 2026-02-21T11:26:01.929 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 253403070470 2026-02-21T11:26:01.929 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:02.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070468 -lt 253403070470 2026-02-21T11:26:02.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:03.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:26:03.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:03.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070468 -lt 253403070470 2026-02-21T11:26:03.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:04.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:26:04.373 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:04.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070471 -lt 253403070470 2026-02-21T11:26:04.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:04.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-292057776131 2026-02-21T11:26:04.589 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:04.590 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:26:04.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-292057776131 2026-02-21T11:26:04.591 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:04.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776131 2026-02-21T11:26:04.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 292057776131' 2026-02-21T11:26:04.593 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 292057776131 2026-02-21T11:26:04.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:26:04.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-21T11:26:04.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:04.820 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117009 2026-02-21T11:26:04.820 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:04.821 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:26:04.822 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117009 2026-02-21T11:26:04.822 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:04.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117009 2026-02-21T11:26:04.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117009' 2026-02-21T11:26:04.823 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117009 2026-02-21T11:26:04.824 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:26:05.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117009 -lt 115964117009 2026-02-21T11:26:05.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:26:05.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:05.044 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:05.323 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:26:05.324 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:26:05.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:26:05.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:26:05.541 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:05.541 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:05.822 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:26:05.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:26:05.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:26:05.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:26:05.823 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:26:06.145 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:26:06.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-1-0 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-1-0 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:26:06.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-1-0 td/test-erasure-eio/COPY 2026-02-21T11:26:06.185 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:26:06.187 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:26:06.188 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:26:06.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-21T11:26:06.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-21T11:26:06.189 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74328-1-256 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74328-1-256 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:26:06.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74328-1-256 td/test-erasure-eio/ORIGINAL 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74328-1-256 td/test-erasure-eio 1 256 add 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-1-256 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-21T11:26:06.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:26:06.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:06.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:26:06.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-1-256 2026-02-21T11:26:06.217 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:06.217 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:26:06.434 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:26:06.434 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:06.435 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:26:06.435 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:26:06.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:26:06.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:26:06.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-21T11:26:06.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:26:06.725 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:26:06.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-21T11:26:06.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:06.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:06.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:06.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:07.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:07.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:07.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:26:07.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:26:07.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:26:07.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:26:07.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:26:07.047 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:07.195 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:07.196 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:07.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:26:07.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:26:07.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:26:07.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:26:07.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:26:07.215 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:07.213+0000 7faa6a026a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:07.216 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:07.213+0000 7faa6a026a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:07.217 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:07.213+0000 7faa6a026a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:07.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:26:07.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:07.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:26:07.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:07.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:07.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:07.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:07.407 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:26:07.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:07.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:07.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:07.628 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:07.625+0000 7faa6a026a40 -1 Falling back to public interface 2026-02-21T11:26:08.250 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:08.245+0000 7faa6a026a40 -1 osd.1 73 log_to_monitors true 2026-02-21T11:26:08.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:08.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:08.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:08.623 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:08.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:08.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:08.839 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,73) [v2:127.0.0.1:6810/2903539562,v1:127.0.0.1:6811/2903539562] [v2:127.0.0.1:6812/2903539562,v1:127.0.0.1:6813/2903539562] exists,up d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:26:08.840 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:26:08.841 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:26:08.841 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:26:08.841 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:26:08.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:26:08.941 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:26:08.941 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:26:08.941 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:26:08.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:26:08.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:26:08.942 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:26:09.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:26:09.154 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:09.154 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:26:09.154 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:26:09.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:26:09.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:09.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:26:09.233 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803799 2026-02-21T11:26:09.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803799 2026-02-21T11:26:09.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799' 2026-02-21T11:26:09.234 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:09.234 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:26:09.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=330712481795 2026-02-21T11:26:09.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 330712481795 2026-02-21T11:26:09.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-330712481795' 2026-02-21T11:26:09.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:09.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:26:09.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776134 2026-02-21T11:26:09.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776134 2026-02-21T11:26:09.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-330712481795 2-292057776134' 2026-02-21T11:26:09.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:09.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:26:09.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117012 2026-02-21T11:26:09.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117012 2026-02-21T11:26:09.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-330712481795 2-292057776134 3-115964117012' 2026-02-21T11:26:09.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:09.472 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803799 2026-02-21T11:26:09.472 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:09.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:26:09.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803799 2026-02-21T11:26:09.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:09.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803799 2026-02-21T11:26:09.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803799' 2026-02-21T11:26:09.476 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803799 2026-02-21T11:26:09.476 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:09.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803799 -lt 25769803799 2026-02-21T11:26:09.694 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:09.694 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-330712481795 2026-02-21T11:26:09.694 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:09.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:26:09.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-330712481795 2026-02-21T11:26:09.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:09.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=330712481795 2026-02-21T11:26:09.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 330712481795' 2026-02-21T11:26:09.698 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 330712481795 2026-02-21T11:26:09.698 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:09.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-21T11:26:09.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:09.915 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-292057776134 2026-02-21T11:26:09.915 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:09.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:26:09.917 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-292057776134 2026-02-21T11:26:09.918 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:09.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776134 2026-02-21T11:26:09.919 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 292057776134' 2026-02-21T11:26:09.919 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 292057776134 2026-02-21T11:26:09.919 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:26:10.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-21T11:26:10.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:10.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117012 2026-02-21T11:26:10.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:10.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:26:10.136 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117012 2026-02-21T11:26:10.136 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:10.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117012 2026-02-21T11:26:10.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117012' 2026-02-21T11:26:10.138 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117012 2026-02-21T11:26:10.138 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:26:10.364 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117013 -lt 115964117012 2026-02-21T11:26:10.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:26:10.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:10.365 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:10.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:26:10.654 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:26:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:26:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:26:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:26:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:26:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:26:10.655 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:26:10.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:26:10.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:26:10.866 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:10.866 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:11.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:26:11.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:26:11.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:26:11.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-21T11:26:11.158 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:26:11.158 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:26:11.158 INFO:tasks.workunit.client.0.trial166.stderr:256 bytes copied, 2.4205e-05 s, 10.6 MB/s 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:11.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:11.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:11.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:11.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:11.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:11.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:11.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:26:11.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:26:11.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:26:11.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:26:11.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:26:11.471 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:11.747 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:11.748 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:26:11.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:26:11.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:26:11.750 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:26:11.750 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:11.751 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:26:11.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:26:11.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:26:11.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:26:11.752 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:26:11.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:11.765+0000 7fb560d0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:11.771 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:11.769+0000 7fb560d0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:11.772 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:11.769+0000 7fb560d0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:11.966 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:26:11.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:11.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:12.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:12.192 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:12.189+0000 7fb560d0aa40 -1 Falling back to public interface 2026-02-21T11:26:12.805 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:12.801+0000 7fb560d0aa40 -1 osd.1 78 log_to_monitors true 2026-02-21T11:26:13.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:13.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:13.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:13.181 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:13.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:13.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [77,78) [v2:127.0.0.1:6810/3476848171,v1:127.0.0.1:6811/3476848171] [v2:127.0.0.1:6812/3476848171,v1:127.0.0.1:6813/3476848171] exists,up d387fe64-1512-4dcd-a262-240489a1b19a 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:26:13.399 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:26:13.400 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:26:13.400 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:26:13.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:26:13.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:26:13.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:26:13.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:26:13.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:26:13.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:26:13.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:26:13.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:26:13.497 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:26:13.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:26:13.712 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:13.712 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:26:13.712 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:26:13.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:26:13.712 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:13.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:26:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803802 2026-02-21T11:26:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803802 2026-02-21T11:26:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802' 2026-02-21T11:26:13.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:13.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:26:13.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=352187318275 2026-02-21T11:26:13.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 352187318275 2026-02-21T11:26:13.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-352187318275' 2026-02-21T11:26:13.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:13.872 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:26:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776137 2026-02-21T11:26:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776137 2026-02-21T11:26:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-352187318275 2-292057776137' 2026-02-21T11:26:13.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:13.950 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:26:14.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117015 2026-02-21T11:26:14.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117015 2026-02-21T11:26:14.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-352187318275 2-292057776137 3-115964117015' 2026-02-21T11:26:14.032 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:14.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803802 2026-02-21T11:26:14.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:14.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:26:14.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803802 2026-02-21T11:26:14.034 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:14.035 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803802 2026-02-21T11:26:14.035 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-21T11:26:14.036 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803802 2026-02-21T11:26:14.036 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:14.252 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803800 -lt 25769803802 2026-02-21T11:26:14.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:15.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:26:15.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:15.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803800 -lt 25769803802 2026-02-21T11:26:15.464 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:16.466 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:26:16.466 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:16.685 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-21T11:26:16.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:16.686 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-352187318275 2026-02-21T11:26:16.686 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:16.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:26:16.688 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-352187318275 2026-02-21T11:26:16.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:16.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=352187318275 2026-02-21T11:26:16.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 352187318275' 2026-02-21T11:26:16.690 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 352187318275 2026-02-21T11:26:16.691 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:16.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 352187318275 -lt 352187318275 2026-02-21T11:26:16.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:16.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-292057776137 2026-02-21T11:26:16.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:16.903 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:26:16.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-292057776137 2026-02-21T11:26:16.904 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:16.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776137 2026-02-21T11:26:16.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 292057776137' 2026-02-21T11:26:16.906 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 292057776137 2026-02-21T11:26:16.906 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:26:17.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776137 -lt 292057776137 2026-02-21T11:26:17.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:17.130 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117015 2026-02-21T11:26:17.130 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:17.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:26:17.132 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117015 2026-02-21T11:26:17.132 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:17.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117015 2026-02-21T11:26:17.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117015' 2026-02-21T11:26:17.134 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117015 2026-02-21T11:26:17.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:26:17.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117016 -lt 115964117015 2026-02-21T11:26:17.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:26:17.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:17.355 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:17.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:26:17.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:26:17.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:26:17.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:26:17.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:26:17.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:26:17.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:26:17.643 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:26:17.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:26:17.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:26:17.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:17.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:18.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:26:18.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:26:18.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:26:18.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:26:18.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:26:18.502 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:26:18.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:18.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:26:18.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:26:18.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-1-256 2026-02-21T11:26:18.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:26:18.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:26:18.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-1-256 td/test-erasure-eio/COPY 2026-02-21T11:26:18.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:26:18.540 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:26:18.542 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74328-1-256 td/test-erasure-eio 2 256 add 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74328-1-256 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-21T11:26:18.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-21T11:26:18.544 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:18.544 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:26:18.544 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74328-1-256 2026-02-21T11:26:18.544 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:18.544 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:26:18.756 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:26:18.756 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:18.757 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:26:18.757 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:26:18.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-21T11:26:18.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-21T11:26:18.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-21T11:26:18.757 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-21T11:26:19.063 INFO:tasks.workunit.client.0.trial166.stderr:noout is set 2026-02-21T11:26:19.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:19.076 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:19.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:19.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:19.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:26:19.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-21T11:26:19.386 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74328-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:26:19.539 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:19.540 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:19.540 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:26:19.541 INFO:tasks.workunit.client.0.trial166.stderr:start osd.2 2026-02-21T11:26:19.542 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:19.542 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:26:19.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:26:19.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:26:19.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:26:19.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:26:19.560 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:19.557+0000 7fc9fb6ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:19.561 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:19.557+0000 7fc9fb6ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:19.562 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:19.557+0000 7fc9fb6ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:19.761 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:19.944 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:19.941+0000 7fc9fb6ada40 -1 Falling back to public interface 2026-02-21T11:26:19.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:20.571 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:20.569+0000 7fc9fb6ada40 -1 osd.2 85 log_to_monitors true 2026-02-21T11:26:20.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:20.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:20.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:20.982 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:20.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:20.982 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 up in weight 1 up_from 91 up_thru 79 down_at 88 last_clean_interval [68,85) [v2:127.0.0.1:6818/140197828,v1:127.0.0.1:6819/140197828] [v2:127.0.0.1:6820/140197828,v1:127.0.0.1:6821/140197828] exists,up f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:26:21.190 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:26:21.191 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:26:21.191 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:26:21.191 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:26:21.191 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:26:21.191 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:26:21.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:26:21.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:26:21.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:26:21.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:26:21.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:26:21.294 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:21.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:26:21.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803805 2026-02-21T11:26:21.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803805 2026-02-21T11:26:21.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805' 2026-02-21T11:26:21.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:21.595 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:26:21.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=352187318278 2026-02-21T11:26:21.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 352187318278 2026-02-21T11:26:21.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-352187318278' 2026-02-21T11:26:21.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:21.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:26:21.758 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=390842023939 2026-02-21T11:26:21.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 390842023939 2026-02-21T11:26:21.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-352187318278 2-390842023939' 2026-02-21T11:26:21.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:21.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:26:21.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117019 2026-02-21T11:26:21.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117019 2026-02-21T11:26:21.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-352187318278 2-390842023939 3-115964117019' 2026-02-21T11:26:21.836 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:21.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803805 2026-02-21T11:26:21.837 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:21.837 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:26:21.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803805 2026-02-21T11:26:21.838 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:21.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803805 2026-02-21T11:26:21.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803805' 2026-02-21T11:26:21.839 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803805 2026-02-21T11:26:21.840 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:22.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803803 -lt 25769803805 2026-02-21T11:26:22.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:23.049 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:26:23.050 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:23.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803803 -lt 25769803805 2026-02-21T11:26:23.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:24.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T11:26:24.265 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:24.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803805 2026-02-21T11:26:24.476 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:24.477 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-352187318278 2026-02-21T11:26:24.477 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:24.478 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:26:24.479 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-352187318278 2026-02-21T11:26:24.479 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:24.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=352187318278 2026-02-21T11:26:24.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 352187318278' 2026-02-21T11:26:24.481 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 352187318278 2026-02-21T11:26:24.481 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:24.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 352187318279 -lt 352187318278 2026-02-21T11:26:24.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:24.701 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-390842023939 2026-02-21T11:26:24.702 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:24.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:26:24.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-390842023939 2026-02-21T11:26:24.704 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:24.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=390842023939 2026-02-21T11:26:24.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 390842023939' 2026-02-21T11:26:24.705 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 390842023939 2026-02-21T11:26:24.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:26:24.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-21T11:26:24.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:24.924 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117019 2026-02-21T11:26:24.924 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:24.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:26:24.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117019 2026-02-21T11:26:24.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:24.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117019 2026-02-21T11:26:24.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117019' 2026-02-21T11:26:24.927 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117019 2026-02-21T11:26:24.928 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:26:25.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117019 -lt 115964117019 2026-02-21T11:26:25.139 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:26:25.139 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:25.139 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:26:25.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:26:25.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:26:25.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:26:25.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:25.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:25.900 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records in 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:1+0 records out 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:256 bytes copied, 2.148e-05 s, 11.9 MB/s 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:26:25.901 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:25.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:26.210 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-21T11:26:26.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74328-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:26.495 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:26:26.496 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:26:26.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:26:26.497 INFO:tasks.workunit.client.0.trial166.stderr:start osd.2 2026-02-21T11:26:26.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:26.499 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:26:26.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:26:26.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:26:26.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:26:26.500 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:26:26.518 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:26.512+0000 7f5023b37a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:26.519 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:26.516+0000 7f5023b37a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:26.520 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:26.516+0000 7f5023b37a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:26.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:26:26.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:26.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:26.704 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:26.904 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:26.900+0000 7f5023b37a40 -1 Falling back to public interface 2026-02-21T11:26:26.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:27.465 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:27.460+0000 7f5023b37a40 -1 osd.2 92 log_to_monitors true 2026-02-21T11:26:27.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:27.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:27.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:27.918 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:27.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:27.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:28.133 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 up in weight 1 up_from 96 up_thru 79 down_at 93 last_clean_interval [91,92) [v2:127.0.0.1:6818/3913135695,v1:127.0.0.1:6819/3913135695] [v2:127.0.0.1:6820/3913135695,v1:127.0.0.1:6821/3913135695] exists,up f438ce0d-4634-48c2-9a9f-68146e3924d4 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:26:28.134 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:26:28.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:26:28.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:26:28.135 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:26:28.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:26:28.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:26:28.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:26:28.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:26:28.231 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:26:28.232 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:28.440 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:26:28.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803809 2026-02-21T11:26:28.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803809 2026-02-21T11:26:28.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809' 2026-02-21T11:26:28.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:28.524 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:26:28.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=352187318282 2026-02-21T11:26:28.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 352187318282 2026-02-21T11:26:28.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-352187318282' 2026-02-21T11:26:28.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:28.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:26:28.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=412316860419 2026-02-21T11:26:28.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 412316860419 2026-02-21T11:26:28.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-352187318282 2-412316860419' 2026-02-21T11:26:28.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:28.679 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:26:28.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117022 2026-02-21T11:26:28.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117022 2026-02-21T11:26:28.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-352187318282 2-412316860419 3-115964117022' 2026-02-21T11:26:28.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:28.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803809 2026-02-21T11:26:28.759 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:28.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:26:28.761 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803809 2026-02-21T11:26:28.761 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:28.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803809 2026-02-21T11:26:28.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803809' 2026-02-21T11:26:28.763 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803809 2026-02-21T11:26:28.763 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:28.970 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803809 2026-02-21T11:26:28.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:26:29.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:26:29.972 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:30.192 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803809 -lt 25769803809 2026-02-21T11:26:30.193 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:30.194 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-352187318282 2026-02-21T11:26:30.194 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:30.194 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:26:30.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-352187318282 2026-02-21T11:26:30.195 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:30.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=352187318282 2026-02-21T11:26:30.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 352187318282' 2026-02-21T11:26:30.197 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 352187318282 2026-02-21T11:26:30.198 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:30.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 352187318282 -lt 352187318282 2026-02-21T11:26:30.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:30.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-412316860419 2026-02-21T11:26:30.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:30.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:26:30.421 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-412316860419 2026-02-21T11:26:30.421 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:30.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=412316860419 2026-02-21T11:26:30.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 412316860419' 2026-02-21T11:26:30.423 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 412316860419 2026-02-21T11:26:30.423 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:26:30.634 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-21T11:26:30.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:30.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117022 2026-02-21T11:26:30.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:30.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:26:30.637 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117022 2026-02-21T11:26:30.637 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:30.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117022 2026-02-21T11:26:30.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117022' 2026-02-21T11:26:30.639 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 115964117022 2026-02-21T11:26:30.639 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:26:30.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117023 -lt 115964117022 2026-02-21T11:26:30.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:26:30.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:30.856 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:31.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:26:31.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:26:31.149 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:26:31.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:26:31.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:26:31.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:26:31.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:26:31.150 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:26:31.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:26:31.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:26:31.360 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:31.360 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:31.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:26:31.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:26:31.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:26:31.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-21T11:26:31.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-21T11:26:31.962 INFO:tasks.workunit.client.0.trial166.stderr:noout is unset 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74328-1-256 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74328-1-256 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:26:31.975 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74328-1-256 td/test-erasure-eio/COPY 2026-02-21T11:26:32.001 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:26:32.002 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:26:32.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:26:32.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:26:32.004 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:26:32.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:26:32.218 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:26:32.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:26:32.504 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:26:32.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:26:32.515 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:26:32.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:26:32.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:26:32.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:32.517 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:32.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:32.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:32.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:32.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:32.827 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:26:32.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:26:32.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:26:32.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:26:32.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:26:32.831 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:26:32.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:32.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:26:32.833 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:26:32.833 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:32.834 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:26:32.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:26:32.835 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:26:32.859 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:26:32.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:32.860 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:32.860 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:26:32.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:26:32.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:26:32.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:26:32.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:26:32.861 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:32.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:32.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:32.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:26:32.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:26:32.866 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:26:32.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:26:32.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:26:32.868 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:26:32.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:32.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:26:32.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:26:32.870 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:32.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:26:32.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:26:32.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:26:32.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:26:32.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:32.873 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:32.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:26:32.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:26:32.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:26:32.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:26:32.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:26:32.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:32.876 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:32.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:26:32.878 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:26:32.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:26:32.901 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:26:32.901 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:32.901 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:32.901 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:32.901 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:32.901 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:32.902 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:32.902 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:26:32.930 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:26:32.931 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:26:32.931 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:26:32.931 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:26:32.931 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:26:32.932 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:26:32.932 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:26:32.932 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:26:32.932 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:26:32.932 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:32.933 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:32.933 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:32.933 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:26:32.933 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:26:32.933 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:26:32.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:26:32.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:26:32.989 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:26:32.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:26:32.990 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:26:32.990 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:26:32.990 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:26:32.990 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:26:32.991 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:32.991 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:32.991 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:32.991 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:26:32.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:26:32.991 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:26:33.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:26:33.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:26:33.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:26:33.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:26:33.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:26:33.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:26:33.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:26:33.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:26:33.154 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:33.155 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:33.155 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:33.155 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:33.155 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:33.155 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:33.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:26:33.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:26:33.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:26:33.175 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:26:33.293 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:26:33.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:26:34.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:26:34.304 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:26:34.353 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:26:34.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:26:34.360 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:26:32.916+0000 7f771fee6ec0 0 load: jerasure load: lrc 2026-02-21T11:26:34.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-21T11:26:34.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-21T11:26:34.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-21T11:26:34.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:26:34.362 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:26:34.362 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:26:34.362 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:26:34.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:26:34.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:26:34.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:34.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:26:34.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:26:34.367 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:34.368 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:26:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:26:34.369 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:26:34.370 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:26:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=95cde620-a031-482c-8f17-ce7849565ea1 2026-02-21T11:26:34.370 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 95cde620-a031-482c-8f17-ce7849565ea1' 2026-02-21T11:26:34.370 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 95cde620-a031-482c-8f17-ce7849565ea1 2026-02-21T11:26:34.370 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:26:34.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBqlplpirWIFhAAiGt0EAt42p1RWnb7gP40/A== 2026-02-21T11:26:34.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBqlplpirWIFhAAiGt0EAt42p1RWnb7gP40/A=="}' 2026-02-21T11:26:34.379 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 95cde620-a031-482c-8f17-ce7849565ea1 -i td/test-erasure-eio/0/new.json 2026-02-21T11:26:34.489 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:34.497 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:26:34.498 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBqlplpirWIFhAAiGt0EAt42p1RWnb7gP40/A== --osd-uuid 95cde620-a031-482c-8f17-ce7849565ea1 2026-02-21T11:26:34.513 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:34.508+0000 7fad216b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:34.514 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:34.512+0000 7fad216b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:34.515 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:34.512+0000 7fad216b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:34.516 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:34.512+0000 7fad216b6a40 -1 bdev(0x55c726f4e400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:26:34.516 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:34.512+0000 7fad216b6a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:26:35.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:26:35.246 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:26:35.247 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:26:35.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:26:35.247 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:26:35.390 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:26:35.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:26:35.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:26:35.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:26:35.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:35.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:26:35.408 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:35.404+0000 7fb9a42d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:35.409 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:35.404+0000 7fb9a42d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:35.410 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:35.404+0000 7fb9a42d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:35.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:26:35.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:35.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:26:35.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:35.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:35.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:35.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:35.509 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:35.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:35.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:26:35.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:35.855 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:35.852+0000 7fb9a42d8a40 -1 Falling back to public interface 2026-02-21T11:26:36.395 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:36.392+0000 7fb9a42d8a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:26:36.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:36.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:36.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:36.729 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:26:36.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:36.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:26:36.936 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:37.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:37.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:37.938 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:26:37.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:26:37.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:37.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:26:38.151 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2407893751,v1:127.0.0.1:6803/2407893751] [v2:127.0.0.1:6804/2407893751,v1:127.0.0.1:6805/2407893751] exists,up 95cde620-a031-482c-8f17-ce7849565ea1 2026-02-21T11:26:38.154 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:26:38.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:38.156 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:38.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:38.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:26:38.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:26:38.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:26:38.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:26:38.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=443d1b3e-a440-4db9-81eb-a2012a13a7fb 2026-02-21T11:26:38.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 443d1b3e-a440-4db9-81eb-a2012a13a7fb' 2026-02-21T11:26:38.159 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 443d1b3e-a440-4db9-81eb-a2012a13a7fb 2026-02-21T11:26:38.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:26:38.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBulplpQ4zzCRAAyFTBHYJ4IASPoZkVyAzmqQ== 2026-02-21T11:26:38.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBulplpQ4zzCRAAyFTBHYJ4IASPoZkVyAzmqQ=="}' 2026-02-21T11:26:38.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 443d1b3e-a440-4db9-81eb-a2012a13a7fb -i td/test-erasure-eio/1/new.json 2026-02-21T11:26:38.390 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:26:38.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:26:38.400 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBulplpQ4zzCRAAyFTBHYJ4IASPoZkVyAzmqQ== --osd-uuid 443d1b3e-a440-4db9-81eb-a2012a13a7fb 2026-02-21T11:26:38.417 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:38.412+0000 7f09b78d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:38.419 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:38.416+0000 7f09b78d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:38.420 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:38.416+0000 7f09b78d3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:38.420 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:38.416+0000 7f09b78d3a40 -1 bdev(0x55fd53334400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:26:38.420 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:38.416+0000 7f09b78d3a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:26:39.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:26:39.005 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:26:39.006 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:26:39.006 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:26:39.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:26:39.298 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:26:39.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:26:39.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:39.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:26:39.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:26:39.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:26:39.321 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:39.316+0000 7f741554ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:39.323 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:39.320+0000 7f741554ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:39.324 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:39.320+0000 7f741554ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:39.507 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:26:39.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:39.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:26:39.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:39.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:39.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:39.515 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:39.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:39.516 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:39.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:39.640 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:39.636+0000 7f741554ea40 -1 Falling back to public interface 2026-02-21T11:26:39.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:40.368 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:40.364+0000 7f741554ea40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:26:40.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:40.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:40.727 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:26:40.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:40.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:40.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:40.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:41.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:41.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:41.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:26:41.939 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:26:41.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:41.939 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:26:42.155 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2464351028,v1:127.0.0.1:6811/2464351028] [v2:127.0.0.1:6812/2464351028,v1:127.0.0.1:6813/2464351028] exists,up 443d1b3e-a440-4db9-81eb-a2012a13a7fb 2026-02-21T11:26:42.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:42.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:42.159 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:42.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:42.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:26:42.162 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:26:42.163 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fe534232-b86d-49d9-9edd-c54f6834596e 2026-02-21T11:26:42.163 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 fe534232-b86d-49d9-9edd-c54f6834596e 2026-02-21T11:26:42.164 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 fe534232-b86d-49d9-9edd-c54f6834596e' 2026-02-21T11:26:42.164 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:26:42.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBylplptYo0ChAA2Fxlj0BKmPDh1h8ecjFwkg== 2026-02-21T11:26:42.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBylplptYo0ChAA2Fxlj0BKmPDh1h8ecjFwkg=="}' 2026-02-21T11:26:42.173 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fe534232-b86d-49d9-9edd-c54f6834596e -i td/test-erasure-eio/2/new.json 2026-02-21T11:26:42.400 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:26:42.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:26:42.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBylplptYo0ChAA2Fxlj0BKmPDh1h8ecjFwkg== --osd-uuid fe534232-b86d-49d9-9edd-c54f6834596e 2026-02-21T11:26:42.428 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:42.424+0000 7f5e6e848a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:42.429 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:42.424+0000 7f5e6e848a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:42.430 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:42.428+0000 7f5e6e848a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:42.431 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:42.428+0000 7f5e6e848a40 -1 bdev(0x55b13bdbe400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:26:42.431 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:42.428+0000 7f5e6e848a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:26:43.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:26:43.042 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:26:43.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:26:43.043 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:26:43.043 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:26:43.334 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:26:43.335 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:26:43.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:43.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:26:43.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:26:43.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:26:43.359 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:43.356+0000 7f60ee7b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:43.361 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:43.356+0000 7f60ee7b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:43.362 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:43.356+0000 7f60ee7b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:43.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:26:43.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:43.552 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:26:43.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:43.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:43.553 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:43.554 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:43.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:43.554 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:43.555 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:43.715 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:43.712+0000 7f60ee7b3a40 -1 Falling back to public interface 2026-02-21T11:26:43.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:44.337 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:44.332+0000 7f60ee7b3a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:26:44.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:44.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:44.766 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:26:44.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:44.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:44.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:26:44.975 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3620363741,v1:127.0.0.1:6819/3620363741] [v2:127.0.0.1:6820/3620363741,v1:127.0.0.1:6821/3620363741] exists,up fe534232-b86d-49d9-9edd-c54f6834596e 2026-02-21T11:26:44.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:44.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:44.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:44.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:26:44.976 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:44.977 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:44.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:44.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:26:44.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:26:44.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:26:44.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:26:44.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=351128c1-60c5-4ada-9478-a93655f35ac5 2026-02-21T11:26:44.980 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 351128c1-60c5-4ada-9478-a93655f35ac5' 2026-02-21T11:26:44.981 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 351128c1-60c5-4ada-9478-a93655f35ac5 2026-02-21T11:26:44.981 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:26:44.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB0lplpqD4UOxAA30BtPGIPr7YiQGEWtQObtQ== 2026-02-21T11:26:44.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB0lplpqD4UOxAA30BtPGIPr7YiQGEWtQObtQ=="}' 2026-02-21T11:26:44.993 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 351128c1-60c5-4ada-9478-a93655f35ac5 -i td/test-erasure-eio/3/new.json 2026-02-21T11:26:45.210 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:26:45.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:26:45.220 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB0lplpqD4UOxAA30BtPGIPr7YiQGEWtQObtQ== --osd-uuid 351128c1-60c5-4ada-9478-a93655f35ac5 2026-02-21T11:26:45.237 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:45.232+0000 7f8ae6356a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:45.238 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:45.236+0000 7f8ae6356a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:45.239 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:45.236+0000 7f8ae6356a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:45.240 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:45.236+0000 7f8ae6356a40 -1 bdev(0x55989f570400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:26:45.240 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:45.236+0000 7f8ae6356a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:26:45.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:26:45.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:26:45.913 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:26:45.914 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:26:45.914 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:26:46.206 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:26:46.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:26:46.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:46.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:26:46.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:26:46.207 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:26:46.230 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:46.228+0000 7f2a345dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:46.232 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:46.228+0000 7f2a345dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:46.233 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:46.228+0000 7f2a345dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:46.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:26:46.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:46.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:26:46.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:46.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:46.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:46.417 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:46.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:46.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:46.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:26:46.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:46.644 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:46.640+0000 7f2a345dfa40 -1 Falling back to public interface 2026-02-21T11:26:47.328 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:47.324+0000 7f2a345dfa40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:26:47.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:47.637 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:47.638 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:26:47.638 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:47.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:47.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:26:47.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:48.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:48.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:48.850 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:26:48.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:26:48.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:48.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:26:49.065 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2028875704,v1:127.0.0.1:6827/2028875704] [v2:127.0.0.1:6828/2028875704,v1:127.0.0.1:6829/2028875704] exists,up 351128c1-60c5-4ada-9478-a93655f35ac5 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:26:49.066 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:49.067 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:49.067 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:49.067 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:26:49.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:26:49.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:26:49.131 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:26:49.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:26:49.139 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:26:35.856+0000 7fb9a42d8a40 0 load: jerasure load: lrc 2026-02-21T11:26:49.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-21T11:26:49.139 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:26:49.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:26:49.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:26:49.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:26:49.777 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:26:49.787 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:26:50.788 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:26:50.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:26:50.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:26:50.789 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:26:50.789 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:26:50.790 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:26:50.790 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:26:50.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:26:50.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:26:50.791 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:26:50.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:26:50.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:26:50.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:26:50.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:26:50.892 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:26:50.892 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:26:51.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:26:51.099 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:51.100 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:26:51.100 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:26:51.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:26:51.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:51.100 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:26:51.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-21T11:26:51.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-21T11:26:51.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-21T11:26:51.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:51.181 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:26:51.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-21T11:26:51.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-21T11:26:51.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-21T11:26:51.259 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:51.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:26:51.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411333 2026-02-21T11:26:51.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411333 2026-02-21T11:26:51.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411333' 2026-02-21T11:26:51.339 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:26:51.339 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:26:51.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149699 2026-02-21T11:26:51.416 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149699 2026-02-21T11:26:51.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411333 3-111669149699' 2026-02-21T11:26:51.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:51.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-21T11:26:51.417 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:51.417 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:26:51.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-21T11:26:51.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:51.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-21T11:26:51.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-21T11:26:51.421 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803781 2026-02-21T11:26:51.421 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:26:51.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-21T11:26:51.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:51.645 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-21T11:26:51.645 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:51.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:26:51.646 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-21T11:26:51.647 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:51.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-21T11:26:51.648 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574852 2026-02-21T11:26:51.649 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-21T11:26:51.649 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:26:51.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-21T11:26:51.866 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:51.866 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411333 2026-02-21T11:26:51.866 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:51.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:26:51.868 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411333 2026-02-21T11:26:51.869 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:51.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411333 2026-02-21T11:26:51.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-21T11:26:51.870 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 77309411333 2026-02-21T11:26:51.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:26:52.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-21T11:26:52.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:26:52.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149699 2026-02-21T11:26:52.090 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:26:52.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:26:52.092 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149699 2026-02-21T11:26:52.092 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:26:52.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149699 2026-02-21T11:26:52.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-21T11:26:52.094 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149699 2026-02-21T11:26:52.094 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:26:52.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-21T11:26:52.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:26:52.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:52.316 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:52.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:26:52.593 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:26:52.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:26:52.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:26:52.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:26:52.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:26:53.067 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74328-0 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74328-0 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:26:53.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74328-0 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:26:53.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74328-0 td/test-erasure-eio/ORIGINAL 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74328-0 td/test-erasure-eio 0 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-74328-0 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-74328-0 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:26:53.087 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-0 2026-02-21T11:26:53.088 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-0 2026-02-21T11:26:53.088 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:26:53.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:26:53.311 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:53.311 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:26:53.311 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:26:53.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-21T11:26:53.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:26:53.311 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:26:53.312 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:26:53.312 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:26:53.314 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:26:53.315 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:26:53.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:26:53.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:26:53.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:26:53.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:26:53.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:26:53.375 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:26:53.375 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:53.375 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:53.375 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:53.376 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:26:53.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:26:53.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-74328-0 0 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74328-0 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74328-0 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:26:53.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74328-0 td/test-erasure-eio/COPY 2026-02-21T11:26:53.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:26:53.454 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:26:53.455 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-21T11:26:53.456 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-21T11:26:53.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-21T11:26:53.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74328-0 td/test-erasure-eio 1 2026-02-21T11:26:53.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-74328-0 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:26:53.459 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-74328-0 2026-02-21T11:26:53.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:26:53.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-0 2026-02-21T11:26:53.460 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-0 2026-02-21T11:26:53.460 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:26:53.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:26:53.668 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:26:53.668 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:26:53.668 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:26:53.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-21T11:26:53.668 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:26:53.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-21T11:26:53.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:26:53.669 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:26:53.671 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:26:53.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:26:53.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:26:53.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:26:53.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-21T11:26:53.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:26:53.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:26:53.733 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:53.734 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:53.734 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:53.734 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:26:53.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:26:53.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-74328-0 1 2026-02-21T11:26:53.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-21T11:26:53.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74328-0 fail 2026-02-21T11:26:53.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:26:53.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:26:53.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74328-0 2026-02-21T11:26:53.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-21T11:26:53.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-21T11:26:53.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-74328-0 td/test-erasure-eio/COPY 2026-02-21T11:26:53.814 INFO:tasks.workunit.client.0.trial166.stderr:error getting pool-jerasure/obj-eio-74328-0: (5) Input/output error 2026-02-21T11:26:53.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-21T11:26:53.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:26:53.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:26:53.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:26:53.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:26:54.069 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:26:54.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:26:54.380 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:26:54.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:26:54.389 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:26:54.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:26:54.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:26:54.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:54.390 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:54.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:54.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:54.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:54.701 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:54.701 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:26:54.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:26:54.703 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:26:54.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:26:54.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:26:54.706 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:26:54.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:54.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:26:54.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:26:54.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:54.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:26:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:26:54.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:26:54.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:26:54.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:54.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:54.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:26:54.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:26:54.725 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:26:54.725 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:26:54.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:26:54.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:26:54.725 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:26:54.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:26:54.728 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:26:54.729 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:26:54.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:26:54.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:26:54.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:26:54.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:26:54.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:54.732 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:26:54.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:26:54.733 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:26:54.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:26:54.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:26:54.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:26:54.736 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:26:54.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:54.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:54.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:26:54.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:26:54.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:26:54.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:26:54.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:26:54.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:54.740 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:54.740 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:26:54.743 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:26:54.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:26:54.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:26:54.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:26:54.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:26:54.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:26:54.744 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:26:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:26:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:26:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:26:54.745 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:26:54.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:26:54.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:54.766 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:54.766 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:54.766 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:54.766 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:54.766 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:54.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:26:54.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:26:54.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:26:54.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:26:54.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:26:54.794 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:26:54.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:26:54.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:26:54.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:26:54.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:26:54.797 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:54.798 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:54.798 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:54.798 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:26:54.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:26:54.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:26:54.850 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:26:54.850 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:26:54.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:26:54.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:26:54.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:26:54.852 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:26:54.852 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:26:54.852 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:26:54.852 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:54.852 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:54.853 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:54.853 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:26:54.853 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:26:54.853 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:26:54.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:26:54.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:26:54.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:26:54.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:26:54.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:26:54.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:26:54.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:26:55.020 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:26:55.021 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:55.021 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:55.021 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:55.021 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:55.021 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:55.021 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:55.022 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:26:55.022 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:26:55.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:26:55.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:26:55.146 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:26:55.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:26:56.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:26:56.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:26:56.159 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:26:56.159 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:26:56.159 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:56.159 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:56.160 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:26:56.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:26:56.160 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:26:56.210 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:26:56.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:26:56.217 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:26:54.780+0000 7fdf8ccd2ec0 0 load: jerasure load: lrc 2026-02-21T11:26:56.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-21T11:26:56.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-21T11:26:56.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-21T11:26:56.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:26:56.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:26:56.219 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:26:56.219 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:26:56.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:26:56.223 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:56.224 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:56.225 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:26:56.226 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:26:56.227 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=22c9c125-1f4b-42bc-8a15-2651b2720457 2026-02-21T11:26:56.227 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 22c9c125-1f4b-42bc-8a15-2651b2720457 2026-02-21T11:26:56.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 22c9c125-1f4b-42bc-8a15-2651b2720457' 2026-02-21T11:26:56.228 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:26:56.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCAlplpxRDvDRAA1Ud955/S5kckRsL8UD3Odg== 2026-02-21T11:26:56.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCAlplpxRDvDRAA1Ud955/S5kckRsL8UD3Odg=="}' 2026-02-21T11:26:56.235 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 22c9c125-1f4b-42bc-8a15-2651b2720457 -i td/test-erasure-eio/0/new.json 2026-02-21T11:26:56.347 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:56.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:26:56.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAlplpxRDvDRAA1Ud955/S5kckRsL8UD3Odg== --osd-uuid 22c9c125-1f4b-42bc-8a15-2651b2720457 2026-02-21T11:26:56.371 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:56.368+0000 7f4dcf88aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:56.373 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:56.368+0000 7f4dcf88aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:56.374 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:56.368+0000 7f4dcf88aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:56.374 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:56.372+0000 7f4dcf88aa40 -1 bdev(0x5560d31da400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:26:56.374 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:56.372+0000 7f4dcf88aa40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:26:57.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:26:57.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:26:57.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:26:57.014 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:26:57.015 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:26:57.126 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:26:57.127 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:26:57.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:26:57.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:26:57.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:26:57.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:26:57.142 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:57.140+0000 7f7db5f9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:57.144 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:57.140+0000 7f7db5f9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:57.145 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:57.140+0000 7f7db5f9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:26:57.270 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:26:57.270 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:57.271 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:26:57.377 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:57.572 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:57.568+0000 7f7db5f9da40 -1 Falling back to public interface 2026-02-21T11:26:58.212 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:26:58.208+0000 7f7db5f9da40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:26:58.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:58.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:58.378 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:26:58.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:26:58.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:58.378 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:26:58.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:26:59.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:26:59.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:26:59.587 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:26:59.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:26:59.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:26:59.588 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:26:59.801 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1272369738,v1:127.0.0.1:6803/1272369738] [v2:127.0.0.1:6804/1272369738,v1:127.0.0.1:6805/1272369738] exists,up 22c9c125-1f4b-42bc-8a15-2651b2720457 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:26:59.802 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:26:59.803 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:26:59.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:26:59.805 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:26:59.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9c670e1d-c253-4fd7-9d81-22ffdaebe0fc 2026-02-21T11:26:59.806 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 9c670e1d-c253-4fd7-9d81-22ffdaebe0fc 2026-02-21T11:26:59.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 9c670e1d-c253-4fd7-9d81-22ffdaebe0fc' 2026-02-21T11:26:59.806 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:26:59.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCDlplp9nnDMBAAFMCYKJxRL6u66QYpqfqGSw== 2026-02-21T11:26:59.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCDlplp9nnDMBAAFMCYKJxRL6u66QYpqfqGSw=="}' 2026-02-21T11:26:59.820 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9c670e1d-c253-4fd7-9d81-22ffdaebe0fc -i td/test-erasure-eio/1/new.json 2026-02-21T11:27:00.062 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:00.072 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:27:00.073 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCDlplp9nnDMBAAFMCYKJxRL6u66QYpqfqGSw== --osd-uuid 9c670e1d-c253-4fd7-9d81-22ffdaebe0fc 2026-02-21T11:27:00.089 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:00.084+0000 7fab064efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:00.090 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:00.088+0000 7fab064efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:00.091 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:00.088+0000 7fab064efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:00.092 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:00.088+0000 7fab064efa40 -1 bdev(0x55ad98994400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:00.092 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:00.088+0000 7fab064efa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:27:00.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:27:00.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:00.766 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:27:00.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:27:00.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:01.056 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:27:01.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:27:01.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:01.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:01.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:01.056 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:01.072 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:01.068+0000 7fe6d32cba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:01.074 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:01.072+0000 7fe6d32cba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:01.075 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:01.072+0000 7fe6d32cba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:01.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:27:01.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:01.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:27:01.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:01.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:01.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:01.266 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:01.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:01.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:01.267 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:01.485 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:01.544 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:01.540+0000 7fe6d32cba40 -1 Falling back to public interface 2026-02-21T11:27:02.104 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:02.100+0000 7fe6d32cba40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:27:02.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:02.487 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:02.487 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:02.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:02.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:02.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:02.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:03.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:03.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:03.708 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:03.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:27:03.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:03.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:03.921 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1994512429,v1:127.0.0.1:6811/1994512429] [v2:127.0.0.1:6812/1994512429,v1:127.0.0.1:6813/1994512429] exists,up 9c670e1d-c253-4fd7-9d81-22ffdaebe0fc 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:03.922 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:03.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:03.924 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:27:03.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:03.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a8d1a604-e485-415f-81c6-76ce346d0e68 2026-02-21T11:27:03.926 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 a8d1a604-e485-415f-81c6-76ce346d0e68 2026-02-21T11:27:03.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 a8d1a604-e485-415f-81c6-76ce346d0e68' 2026-02-21T11:27:03.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:03.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCHlplpQ8rHNxAApObPzy184HrjVi2yRtISCA== 2026-02-21T11:27:03.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCHlplpQ8rHNxAApObPzy184HrjVi2yRtISCA=="}' 2026-02-21T11:27:03.937 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a8d1a604-e485-415f-81c6-76ce346d0e68 -i td/test-erasure-eio/2/new.json 2026-02-21T11:27:04.164 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:04.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:27:04.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCHlplpQ8rHNxAApObPzy184HrjVi2yRtISCA== --osd-uuid a8d1a604-e485-415f-81c6-76ce346d0e68 2026-02-21T11:27:04.192 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:04.188+0000 7f4e588c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:04.193 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:04.188+0000 7f4e588c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:04.194 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:04.192+0000 7f4e588c1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:04.195 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:04.192+0000 7f4e588c1a40 -1 bdev(0x55c962d82400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:04.195 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:04.192+0000 7f4e588c1a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:27:04.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:27:04.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:04.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:27:04.806 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:27:04.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:05.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:27:05.114 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:27:05.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:05.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:05.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:05.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:05.139 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:05.136+0000 7f7527fc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:05.141 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:05.136+0000 7f7527fc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:05.142 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:05.136+0000 7f7527fc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:05.319 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:05.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:05.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:05.320 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:27:05.535 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:05.551 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:05.548+0000 7f7527fc3a40 -1 Falling back to public interface 2026-02-21T11:27:06.164 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:06.160+0000 7f7527fc3a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:27:06.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:06.537 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:06.537 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:06.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:06.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:06.538 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:27:06.758 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3673492723,v1:127.0.0.1:6819/3673492723] [v2:127.0.0.1:6820/3673492723,v1:127.0.0.1:6821/3673492723] exists,up a8d1a604-e485-415f-81c6-76ce346d0e68 2026-02-21T11:27:06.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:06.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:06.762 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:06.763 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:06.764 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:27:06.765 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:06.766 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c354b8ae-f568-45e4-8a48-05048f30d500 2026-02-21T11:27:06.766 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 c354b8ae-f568-45e4-8a48-05048f30d500 2026-02-21T11:27:06.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c354b8ae-f568-45e4-8a48-05048f30d500' 2026-02-21T11:27:06.767 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:06.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCKlplpw6wfLhAAs7V9Hps2lUtyh0iLwvCHNA== 2026-02-21T11:27:06.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCKlplpw6wfLhAAs7V9Hps2lUtyh0iLwvCHNA=="}' 2026-02-21T11:27:06.775 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c354b8ae-f568-45e4-8a48-05048f30d500 -i td/test-erasure-eio/3/new.json 2026-02-21T11:27:07.002 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:27:07.013 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:27:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCKlplpw6wfLhAAs7V9Hps2lUtyh0iLwvCHNA== --osd-uuid c354b8ae-f568-45e4-8a48-05048f30d500 2026-02-21T11:27:07.031 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:07.028+0000 7f08ba300a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:07.033 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:07.028+0000 7f08ba300a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:07.034 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:07.028+0000 7f08ba300a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:07.034 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:07.032+0000 7f08ba300a40 -1 bdev(0x55f0eac92400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:07.034 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:07.032+0000 7f08ba300a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:27:07.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:27:07.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:07.710 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:27:07.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:27:07.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:08.008 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:27:08.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:27:08.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:08.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:08.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:08.009 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:08.031 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:08.028+0000 7f802da40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:08.033 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:08.028+0000 7f802da40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:08.034 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:08.032+0000 7f802da40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:08.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:27:08.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:08.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:27:08.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:08.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:08.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:08.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:08.216 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:08.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:08.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:27:08.412 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:08.408+0000 7f802da40a40 -1 Falling back to public interface 2026-02-21T11:27:08.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:09.016 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:09.012+0000 7f802da40a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:27:09.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:09.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:09.438 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:09.438 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:09.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:09.439 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:27:09.657 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2401637578,v1:127.0.0.1:6827/2401637578] [v2:127.0.0.1:6828/2401637578,v1:127.0.0.1:6829/2401637578] exists,up c354b8ae-f568-45e4-8a48-05048f30d500 2026-02-21T11:27:09.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:09.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:09.657 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:09.657 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:27:09.657 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:27:09.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:27:09.658 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:09.658 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:09.658 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:09.658 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:27:09.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:27:09.659 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:27:09.722 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:27:09.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:27:09.729 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:26:57.572+0000 7f7db5f9da40 0 load: jerasure load: lrc 2026-02-21T11:27:09.730 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:27:09.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:27:10.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:27:10.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:27:10.396 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:27:10.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:27:11.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:27:11.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:27:11.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:27:11.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:27:11.408 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:27:11.409 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:27:11.409 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:27:11.409 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:27:11.410 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:27:11.410 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:27:11.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:27:11.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:27:11.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:27:11.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:27:11.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:27:11.512 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:11.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:27:11.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-21T11:27:11.806 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-21T11:27:11.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-21T11:27:11.807 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:11.807 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:27:11.884 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-21T11:27:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-21T11:27:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-21T11:27:11.885 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:11.885 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:27:11.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378627 2026-02-21T11:27:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378627 2026-02-21T11:27:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378627' 2026-02-21T11:27:11.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:11.966 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:27:12.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149699 2026-02-21T11:27:12.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149699 2026-02-21T11:27:12.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378627 3-111669149699' 2026-02-21T11:27:12.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:12.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-21T11:27:12.045 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:12.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:27:12.047 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-21T11:27:12.047 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:12.048 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-21T11:27:12.048 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803781 2026-02-21T11:27:12.049 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-21T11:27:12.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:12.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-21T11:27:12.276 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:27:13.277 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:27:13.277 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:13.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-21T11:27:13.486 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:13.487 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-21T11:27:13.487 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:13.488 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:27:13.489 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-21T11:27:13.489 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:13.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-21T11:27:13.491 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574852 2026-02-21T11:27:13.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-21T11:27:13.491 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:27:13.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574853 -lt 55834574852 2026-02-21T11:27:13.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:13.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378627 2026-02-21T11:27:13.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:13.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:27:13.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378627 2026-02-21T11:27:13.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:13.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378627 2026-02-21T11:27:13.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378627' 2026-02-21T11:27:13.710 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378627 2026-02-21T11:27:13.711 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:27:13.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378628 -lt 81604378627 2026-02-21T11:27:13.921 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:13.922 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149699 2026-02-21T11:27:13.922 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:13.923 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:27:13.923 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149699 2026-02-21T11:27:13.923 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:13.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149699 2026-02-21T11:27:13.925 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-21T11:27:13.925 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149699 2026-02-21T11:27:13.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:27:14.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-21T11:27:14.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:27:14.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:14.158 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:14.452 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:27:14.453 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:27:14.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:27:14.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:27:14.453 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:27:14.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:27:14.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:27:14.454 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:27:14.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:27:14.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:27:14.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:14.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-21T11:27:14.946 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74328-1 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74328-1 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74328-1 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:27:14.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74328-1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74328-1 td/test-erasure-eio 1 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-74328-1 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-74328-1 2026-02-21T11:27:14.967 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:27:14.968 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-1 2026-02-21T11:27:14.968 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-1 2026-02-21T11:27:14.968 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:27:15.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:27:15.183 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:15.183 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:27:15.183 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:27:15.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-21T11:27:15.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:27:15.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-21T11:27:15.184 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:27:15.184 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:27:15.186 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-21T11:27:15.187 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:27:15.187 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:27:15.187 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:15.187 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:15.187 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:15.187 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:27:15.188 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:27:15.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:27:15.253 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:27:15.254 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:27:15.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-21T11:27:15.254 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:27:15.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:27:15.255 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:15.255 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:15.255 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:15.255 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:27:15.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:27:15.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-74328-1 1 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74328-1 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74328-1 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:27:15.315 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74328-1 td/test-erasure-eio/COPY 2026-02-21T11:27:15.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:27:15.336 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:27:15.337 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-21T11:27:15.338 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74328-1 td/test-erasure-eio 2 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:27:15.340 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-74328-1 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=2 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-74328-1 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:27:15.341 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-1 2026-02-21T11:27:15.342 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-1 2026-02-21T11:27:15.342 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:27:15.558 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:27:15.558 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:15.558 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:27:15.558 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:27:15.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-21T11:27:15.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:27:15.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=2 2026-02-21T11:27:15.559 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:27:15.560 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/2/type 2026-02-21T11:27:15.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:27:15.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-21T11:27:15.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:27:15.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-21T11:27:15.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:27:15.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:15.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.2.asok 2026-02-21T11:27:15.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:27:15.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:27:15.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:27:15.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:27:15.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.2 2026-02-21T11:27:15.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T11:27:15.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T11:27:15.624 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:15.624 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:15.624 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:15.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.2.asok 2026-02-21T11:27:15.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:27:15.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-74328-1 2 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74328-1 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74328-1 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:27:15.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:27:15.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74328-1 td/test-erasure-eio/COPY 2026-02-21T11:27:15.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:27:15.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:27:15.705 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:27:15.706 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:27:15.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:27:15.707 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:27:15.964 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:27:15.973 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:27:16.270 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:27:16.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:27:16.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:27:16.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:27:16.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:27:16.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:27:16.282 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:27:16.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:27:16.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:27:16.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:27:16.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:27:16.595 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:27:16.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:27:16.596 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:27:16.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:27:16.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:27:16.599 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:27:16.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:16.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:27:16.600 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:27:16.600 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:16.601 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:27:16.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:27:16.603 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:27:16.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:27:16.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.616 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:27:16.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:27:16.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:27:16.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:27:16.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:27:16.623 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:27:16.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:27:16.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:27:16.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:27:16.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:16.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:27:16.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:27:16.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:16.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:27:16.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:27:16.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:27:16.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:27:16.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:27:16.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:27:16.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:27:16.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:27:16.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:27:16.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:27:16.635 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:27:16.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:27:16.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:27:16.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:27:16.635 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:27:16.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:27:16.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:27:16.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:27:16.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:27:16.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:27:16.636 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.659 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:16.660 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:27:16.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:27:16.689 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:27:16.690 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:27:16.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:27:16.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:27:16.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:27:16.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:27:16.752 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:27:16.753 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:27:16.754 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:27:16.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:27:16.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:27:16.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:27:16.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:27:16.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:27:16.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:27:16.809 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:27:16.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:27:16.925 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:16.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:16.926 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:16.926 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:16.926 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:16.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:16.926 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:27:16.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:27:16.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:27:16.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:27:17.058 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:27:17.066 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:27:18.068 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:27:18.117 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:27:18.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:27:18.125 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:27:16.676+0000 7f8b48119ec0 0 load: jerasure load: lrc 2026-02-21T11:27:18.125 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-21T11:27:18.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-21T11:27:18.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-21T11:27:18.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:27:18.126 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:27:18.127 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:27:18.127 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:27:18.130 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:18.131 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:18.132 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:18.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:18.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:27:18.133 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:18.133 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4c927c72-9a67-403d-ae95-89ea622eb132 2026-02-21T11:27:18.134 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4c927c72-9a67-403d-ae95-89ea622eb132' 2026-02-21T11:27:18.134 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 4c927c72-9a67-403d-ae95-89ea622eb132 2026-02-21T11:27:18.134 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:18.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCWlplpRw9rCBAAebF1zdfJVDQj1dfKzWcCDA== 2026-02-21T11:27:18.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCWlplpRw9rCBAAebF1zdfJVDQj1dfKzWcCDA=="}' 2026-02-21T11:27:18.142 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4c927c72-9a67-403d-ae95-89ea622eb132 -i td/test-erasure-eio/0/new.json 2026-02-21T11:27:18.254 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:18.261 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:27:18.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWlplpRw9rCBAAebF1zdfJVDQj1dfKzWcCDA== --osd-uuid 4c927c72-9a67-403d-ae95-89ea622eb132 2026-02-21T11:27:18.279 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:18.276+0000 7f9069fe5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:18.280 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:18.276+0000 7f9069fe5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:18.281 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:18.276+0000 7f9069fe5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:18.281 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:18.276+0000 7f9069fe5a40 -1 bdev(0x559a1f890400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:18.282 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:18.276+0000 7f9069fe5a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:27:18.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:27:18.997 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:18.998 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:27:18.999 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:27:18.999 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:19.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:27:19.108 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:27:19.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:19.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:19.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:19.109 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:19.123 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:19.120+0000 7f15aea59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:19.124 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:19.120+0000 7f15aea59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:19.127 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:19.120+0000 7f15aea59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:19.255 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:19.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:19.256 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:27:19.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:19.536 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:19.532+0000 7f15aea59a40 -1 Falling back to public interface 2026-02-21T11:27:20.081 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:20.076+0000 7f15aea59a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:27:20.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:20.365 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:20.366 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:20.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:20.366 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:20.367 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:27:20.572 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:21.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:21.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:21.574 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:21.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:27:21.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:21.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:27:21.787 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3384612424,v1:127.0.0.1:6803/3384612424] [v2:127.0.0.1:6804/3384612424,v1:127.0.0.1:6805/3384612424] exists,up 4c927c72-9a67-403d-ae95-89ea622eb132 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:27:21.791 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:21.792 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:21.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:21.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:21.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:21.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:21.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:21.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:21.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:27:21.795 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:21.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7aa026ee-fce1-422a-a8d0-e91b8aeba1a2 2026-02-21T11:27:21.795 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 7aa026ee-fce1-422a-a8d0-e91b8aeba1a2 2026-02-21T11:27:21.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7aa026ee-fce1-422a-a8d0-e91b8aeba1a2' 2026-02-21T11:27:21.796 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:21.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZlplp7YjmLxAAH1ubAazKT+4TiJlrThUmZQ== 2026-02-21T11:27:21.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZlplp7YjmLxAAH1ubAazKT+4TiJlrThUmZQ=="}' 2026-02-21T11:27:21.805 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7aa026ee-fce1-422a-a8d0-e91b8aeba1a2 -i td/test-erasure-eio/1/new.json 2026-02-21T11:27:22.015 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:22.025 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:27:22.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZlplp7YjmLxAAH1ubAazKT+4TiJlrThUmZQ== --osd-uuid 7aa026ee-fce1-422a-a8d0-e91b8aeba1a2 2026-02-21T11:27:22.043 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:22.040+0000 7f9038ce8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:22.045 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:22.040+0000 7f9038ce8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:22.046 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:22.040+0000 7f9038ce8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:22.046 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:22.044+0000 7f9038ce8a40 -1 bdev(0x559171880400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:22.047 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:22.044+0000 7f9038ce8a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:27:22.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:27:22.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:22.698 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:27:22.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:27:22.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:22.980 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:27:22.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:27:22.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:22.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:22.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:22.981 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:23.004 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:23.000+0000 7f51ae6bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:23.006 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:23.000+0000 7f51ae6bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:23.007 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:23.004+0000 7f51ae6bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:23.196 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:27:23.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:23.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:27:23.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:23.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:23.197 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:23.197 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:23.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:23.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:23.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:23.352 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:23.348+0000 7f51ae6bfa40 -1 Falling back to public interface 2026-02-21T11:27:23.410 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:23.924 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:23.920+0000 7f51ae6bfa40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:27:24.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:24.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:24.411 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:24.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:24.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:24.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:24.625 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2540408405,v1:127.0.0.1:6811/2540408405] [v2:127.0.0.1:6812/2540408405,v1:127.0.0.1:6813/2540408405] exists,up 7aa026ee-fce1-422a-a8d0-e91b8aeba1a2 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:24.628 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:24.629 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:24.630 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:24.631 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:27:24.632 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:24.632 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b032a087-71c5-462c-89a1-4652000fe715 2026-02-21T11:27:24.632 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 b032a087-71c5-462c-89a1-4652000fe715 2026-02-21T11:27:24.633 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b032a087-71c5-462c-89a1-4652000fe715' 2026-02-21T11:27:24.633 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:24.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCclplpKcQnJhAA4IdfWGqonQXXKLFJDdUfQA== 2026-02-21T11:27:24.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCclplpKcQnJhAA4IdfWGqonQXXKLFJDdUfQA=="}' 2026-02-21T11:27:24.642 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b032a087-71c5-462c-89a1-4652000fe715 -i td/test-erasure-eio/2/new.json 2026-02-21T11:27:24.863 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:24.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:27:24.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCclplpKcQnJhAA4IdfWGqonQXXKLFJDdUfQA== --osd-uuid b032a087-71c5-462c-89a1-4652000fe715 2026-02-21T11:27:24.900 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:24.896+0000 7f1e2fd0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:24.901 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:24.896+0000 7f1e2fd0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:24.902 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:24.896+0000 7f1e2fd0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:24.903 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:24.900+0000 7f1e2fd0ca40 -1 bdev(0x5567b7db4400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:24.903 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:24.900+0000 7f1e2fd0ca40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:27:25.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:27:25.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:25.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:27:25.567 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:27:25.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:25.858 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:27:25.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:27:25.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:25.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:25.858 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:25.859 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:25.883 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:25.880+0000 7fc9f8c94a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:25.884 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:25.880+0000 7fc9f8c94a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:25.886 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:25.880+0000 7fc9f8c94a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:26.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:27:26.079 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:26.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:27:26.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:26.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:26.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:26.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:26.080 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:26.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:26.081 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:27:26.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:26.344 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:26.340+0000 7fc9f8c94a40 -1 Falling back to public interface 2026-02-21T11:27:26.872 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:26.867+0000 7fc9f8c94a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:27:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:27.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:27.288 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:27.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:27.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:27.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:27:27.504 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1908540045,v1:127.0.0.1:6819/1908540045] [v2:127.0.0.1:6820/1908540045,v1:127.0.0.1:6821/1908540045] exists,up b032a087-71c5-462c-89a1-4652000fe715 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:27.508 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:27.509 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:27.510 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:27.511 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=108e8fd6-c2a1-4eb9-a367-3c1bd1d6a95b 2026-02-21T11:27:27.512 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 108e8fd6-c2a1-4eb9-a367-3c1bd1d6a95b 2026-02-21T11:27:27.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 108e8fd6-c2a1-4eb9-a367-3c1bd1d6a95b' 2026-02-21T11:27:27.512 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:27.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCflplpI2D0HhAA/MapoHl5qL4OXjp2NfyIjw== 2026-02-21T11:27:27.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCflplpI2D0HhAA/MapoHl5qL4OXjp2NfyIjw=="}' 2026-02-21T11:27:27.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 108e8fd6-c2a1-4eb9-a367-3c1bd1d6a95b -i td/test-erasure-eio/3/new.json 2026-02-21T11:27:27.738 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:27:27.748 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:27:27.749 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCflplpI2D0HhAA/MapoHl5qL4OXjp2NfyIjw== --osd-uuid 108e8fd6-c2a1-4eb9-a367-3c1bd1d6a95b 2026-02-21T11:27:27.767 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:27.763+0000 7fc603aa2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:27.768 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:27.763+0000 7fc603aa2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:27.769 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:27.763+0000 7fc603aa2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:27.770 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:27.763+0000 7fc603aa2a40 -1 bdev(0x55c34d57c400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:27.770 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:27.763+0000 7fc603aa2a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:27:28.420 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:27:28.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:28.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:27:28.421 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:27:28.421 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:28.713 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:27:28.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:27:28.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:28.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:28.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:28.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:28.738 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:28.735+0000 7f67a1473a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:28.740 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:28.735+0000 7f67a1473a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:28.741 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:28.735+0000 7f67a1473a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:28.926 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:28.927 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:28.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:28.928 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:27:29.107 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:29.103+0000 7f67a1473a40 -1 Falling back to public interface 2026-02-21T11:27:29.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:29.757 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:29.751+0000 7f67a1473a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:27:30.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:30.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:30.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:30.137 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:30.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:30.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:27:30.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:31.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:31.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:31.354 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:31.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:27:31.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:31.355 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:27:31.560 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3729139991,v1:127.0.0.1:6827/3729139991] [v2:127.0.0.1:6828/3729139991,v1:127.0.0.1:6829/3729139991] exists,up 108e8fd6-c2a1-4eb9-a367-3c1bd1d6a95b 2026-02-21T11:27:31.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:31.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:31.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:31.561 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:27:31.561 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:27:31.561 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:27:31.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:31.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:31.562 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:31.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:27:31.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:27:31.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:27:31.616 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:27:31.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:27:31.623 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:27:19.536+0000 7f15aea59a40 0 load: jerasure load: lrc 2026-02-21T11:27:31.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-21T11:27:31.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:27:31.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:27:31.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:27:31.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:27:32.286 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:27:32.297 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:27:33.299 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:27:33.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:27:33.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:27:33.300 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:27:33.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:27:33.301 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:27:33.301 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:27:33.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:27:33.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:27:33.301 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:27:33.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:27:33.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:27:33.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:27:33.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:27:33.412 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:27:33.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:27:33.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:27:33.625 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:33.625 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:27:33.625 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:27:33.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:27:33.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:33.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:27:33.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803782 2026-02-21T11:27:33.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803782 2026-02-21T11:27:33.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782' 2026-02-21T11:27:33.703 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:33.703 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:27:33.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607556 2026-02-21T11:27:33.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607556 2026-02-21T11:27:33.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607556' 2026-02-21T11:27:33.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:33.785 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:27:33.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444036 2026-02-21T11:27:33.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444036 2026-02-21T11:27:33.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607556 2-73014444036' 2026-02-21T11:27:33.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:33.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:27:33.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182403 2026-02-21T11:27:33.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182403 2026-02-21T11:27:33.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607556 2-73014444036 3-107374182403' 2026-02-21T11:27:33.943 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:33.943 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803782 2026-02-21T11:27:33.943 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:33.945 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:27:33.945 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803782 2026-02-21T11:27:33.946 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:33.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803782 2026-02-21T11:27:33.947 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-21T11:27:33.947 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803782 2026-02-21T11:27:33.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:34.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-21T11:27:34.167 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:27:35.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:27:35.168 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:35.382 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-21T11:27:35.383 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:35.383 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607556 2026-02-21T11:27:35.383 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:35.385 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:27:35.385 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607556 2026-02-21T11:27:35.386 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:35.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607556 2026-02-21T11:27:35.387 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-21T11:27:35.387 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607556 2026-02-21T11:27:35.387 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:27:35.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607557 -lt 51539607556 2026-02-21T11:27:35.609 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:35.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444036 2026-02-21T11:27:35.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:35.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:27:35.612 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444036 2026-02-21T11:27:35.612 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:35.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444036 2026-02-21T11:27:35.614 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444036 2026-02-21T11:27:35.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-21T11:27:35.614 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:27:35.839 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-21T11:27:35.840 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:35.840 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182403 2026-02-21T11:27:35.840 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:35.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:27:35.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182403 2026-02-21T11:27:35.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:35.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182403 2026-02-21T11:27:35.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-21T11:27:35.844 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 107374182403 2026-02-21T11:27:35.844 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:27:36.069 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-21T11:27:36.069 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:27:36.070 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:36.071 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:36.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:27:36.351 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:27:36.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:27:36.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:27:36.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:27:36.351 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:27:36.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:27:36.352 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:27:36.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:27:36.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:27:36.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:36.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:36.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-74328-1 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-74328-1 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:27:36.854 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-74328-1 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:27:36.855 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-74328-1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:27:36.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-74328-1 td/test-erasure-eio 1 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-74328-1 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-74328-1 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:27:36.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-74328-1 2026-02-21T11:27:36.875 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-74328-1 2026-02-21T11:27:36.875 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:27:37.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-74328-1 remove 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-74328-1 remove 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:27:37.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:27:37.084 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:27:37.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-74328-1 remove 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-21T11:27:37.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-74328-1 remove 2026-02-21T11:27:37.594 INFO:tasks.workunit.client.0.trial166.stdout:remove 1#2:dce1ad5e:::obj-remove-74328-1:head# 2026-02-21T11:27:37.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-21T11:27:37.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:37.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:27:37.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T11:27:37.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:27:37.695 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:37.696 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:37.697 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:37.697 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:37.697 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:37.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:37.697 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:start osd.1 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:37.698 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-21T11:27:37.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T11:27:37.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:27:37.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:27:37.699 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:27:37.715 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:37.711+0000 7fdfaabdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:37.716 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:37.711+0000 7fdfaabdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:37.717 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:37.711+0000 7fdfaabdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:37.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T11:27:37.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:37.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:27:37.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:37.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:37.915 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:37.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:37.916 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:27:37.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:37.916 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:38.121 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:38.195 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:38.191+0000 7fdfaabdaa40 -1 Falling back to public interface 2026-02-21T11:27:38.800 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:38.795+0000 7fdfaabdaa40 -1 osd.1 33 log_to_monitors true 2026-02-21T11:27:39.122 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:39.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:39.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:39.123 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:39.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:39.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:39.330 INFO:tasks.workunit.client.0.trial166.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [12,33) [v2:127.0.0.1:6810/2419468410,v1:127.0.0.1:6811/2419468410] [v2:127.0.0.1:6812/2419468410,v1:127.0.0.1:6813/2419468410] exists,up 7aa026ee-fce1-422a-a8d0-e91b8aeba1a2 2026-02-21T11:27:39.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:39.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:39.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:39.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:27:39.330 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:27:39.331 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:27:39.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:27:39.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:27:39.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:27:39.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:27:39.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:27:39.435 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:27:39.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:27:39.653 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:39.653 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:27:39.653 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:27:39.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:27:39.653 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:39.654 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:27:39.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803785 2026-02-21T11:27:39.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803785 2026-02-21T11:27:39.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785' 2026-02-21T11:27:39.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:39.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:27:39.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789955 2026-02-21T11:27:39.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789955 2026-02-21T11:27:39.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-158913789955' 2026-02-21T11:27:39.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:39.811 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:27:39.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444039 2026-02-21T11:27:39.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444039 2026-02-21T11:27:39.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-158913789955 2-73014444039' 2026-02-21T11:27:39.887 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:39.887 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:27:39.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182406 2026-02-21T11:27:39.968 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182406 2026-02-21T11:27:39.968 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-158913789955 2-73014444039 3-107374182406' 2026-02-21T11:27:39.968 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:39.968 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803785 2026-02-21T11:27:39.968 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:39.969 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:27:39.970 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803785 2026-02-21T11:27:39.970 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:39.971 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803785 2026-02-21T11:27:39.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-21T11:27:39.972 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803785 2026-02-21T11:27:39.972 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:40.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-21T11:27:40.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:27:41.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:27:41.192 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:41.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803785 2026-02-21T11:27:41.409 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:41.410 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-158913789955 2026-02-21T11:27:41.410 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:41.411 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:27:41.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-158913789955 2026-02-21T11:27:41.412 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:41.413 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789955 2026-02-21T11:27:41.414 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 158913789955' 2026-02-21T11:27:41.414 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 158913789955 2026-02-21T11:27:41.414 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:27:41.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-21T11:27:41.639 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:41.640 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444039 2026-02-21T11:27:41.640 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:41.641 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:27:41.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444039 2026-02-21T11:27:41.642 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:41.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444039 2026-02-21T11:27:41.644 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444039' 2026-02-21T11:27:41.644 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 73014444039 2026-02-21T11:27:41.644 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:27:41.862 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-21T11:27:41.863 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:41.863 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182406 2026-02-21T11:27:41.863 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:41.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:27:41.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182406 2026-02-21T11:27:41.865 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:41.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182406 2026-02-21T11:27:41.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-21T11:27:41.867 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 107374182406 2026-02-21T11:27:41.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:27:42.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182407 -lt 107374182406 2026-02-21T11:27:42.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:27:42.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:42.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:42.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:27:42.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:27:42.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:27:42.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:27:42.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:27:42.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:27:42.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:27:42.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:27:42.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:27:42.567 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:27:42.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:42.568 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:27:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:27:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:27:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-74328-1 2026-02-21T11:27:42.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:27:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:27:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-74328-1 2026-02-21T11:27:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:27:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:27:42.851 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-74328-1 td/test-erasure-eio/COPY 2026-02-21T11:27:42.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:27:42.869 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:27:42.870 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-21T11:27:42.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-74328-1 td/test-erasure-eio 2 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-74328-1 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-21T11:27:42.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-74328-1 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-74328-1 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-74328-1 2026-02-21T11:27:42.874 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:27:43.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:27:43.082 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:43.082 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-74328-1 remove 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-74328-1 remove 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:27:43.083 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:27:43.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-74328-1 remove 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-21T11:27:43.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-74328-1 remove 2026-02-21T11:27:43.566 INFO:tasks.workunit.client.0.trial166.stdout:remove 2#2:dce1ad5e:::obj-remove-74328-1:head# 2026-02-21T11:27:43.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-21T11:27:43.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:43.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:27:43.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:27:43.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:27:43.662 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:43.663 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:43.664 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:43.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:27:43.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:27:43.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:27:43.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:27:43.665 INFO:tasks.workunit.client.0.trial166.stderr:start osd.2 2026-02-21T11:27:43.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:43.666 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:27:43.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:27:43.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:27:43.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:27:43.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:27:43.678 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:43.675+0000 7f5e2cedaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:43.680 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:43.675+0000 7f5e2cedaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:43.680 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:43.675+0000 7f5e2cedaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:43.879 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:43.880 INFO:tasks.workunit.client.0.trial166.stderr:0 2026-02-21T11:27:43.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:43.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:27:44.089 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:44.123 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:44.119+0000 7f5e2cedaa40 -1 Falling back to public interface 2026-02-21T11:27:44.752 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:44.747+0000 7f5e2cedaa40 -1 osd.2 40 log_to_monitors true 2026-02-21T11:27:45.090 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:45.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:45.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:45.091 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:45.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:45.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:27:45.302 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 up in weight 1 up_from 43 up_thru 34 down_at 40 last_clean_interval [17,40) [v2:127.0.0.1:6818/196407204,v1:127.0.0.1:6819/196407204] [v2:127.0.0.1:6820/196407204,v1:127.0.0.1:6821/196407204] exists,up b032a087-71c5-462c-89a1-4652000fe715 2026-02-21T11:27:45.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:45.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:45.302 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:45.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-21T11:27:45.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:27:45.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:27:45.303 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:27:45.303 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:27:45.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:27:45.304 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:27:45.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:27:45.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:27:45.304 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:27:45.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:27:45.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:27:45.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:27:45.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:27:45.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:27:45.405 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:27:45.626 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:27:45.626 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:27:45.626 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:27:45.627 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:27:45.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:27:45.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:45.627 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:27:45.709 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803788 2026-02-21T11:27:45.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803788 2026-02-21T11:27:45.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788' 2026-02-21T11:27:45.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:45.710 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:27:45.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789958 2026-02-21T11:27:45.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789958 2026-02-21T11:27:45.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-158913789958' 2026-02-21T11:27:45.790 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:45.790 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:27:45.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593731 2026-02-21T11:27:45.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593731 2026-02-21T11:27:45.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-158913789958 2-184683593731' 2026-02-21T11:27:45.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:27:45.868 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:27:45.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182410 2026-02-21T11:27:45.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182410 2026-02-21T11:27:45.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-158913789958 2-184683593731 3-107374182410' 2026-02-21T11:27:45.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:45.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803788 2026-02-21T11:27:45.948 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:45.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:27:45.950 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803788 2026-02-21T11:27:45.951 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:45.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803788 2026-02-21T11:27:45.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-21T11:27:45.952 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.0 seq 25769803788 2026-02-21T11:27:45.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:46.176 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-21T11:27:46.177 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:27:47.178 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:27:47.179 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:27:47.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803789 -lt 25769803788 2026-02-21T11:27:47.388 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:47.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-158913789958 2026-02-21T11:27:47.389 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:47.390 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:27:47.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-158913789958 2026-02-21T11:27:47.391 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:47.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789958 2026-02-21T11:27:47.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 158913789958' 2026-02-21T11:27:47.393 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.1 seq 158913789958 2026-02-21T11:27:47.393 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:27:47.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789958 -lt 158913789958 2026-02-21T11:27:47.621 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:47.622 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-184683593731 2026-02-21T11:27:47.622 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:47.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:27:47.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-184683593731 2026-02-21T11:27:47.624 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593731 2026-02-21T11:27:47.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-21T11:27:47.626 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.2 seq 184683593731 2026-02-21T11:27:47.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:27:47.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-21T11:27:47.848 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:27:47.849 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182410 2026-02-21T11:27:47.849 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:27:47.850 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:27:47.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182410 2026-02-21T11:27:47.851 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:27:47.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182410 2026-02-21T11:27:47.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182410' 2026-02-21T11:27:47.853 INFO:tasks.workunit.client.0.trial166.stderr:waiting osd.3 seq 107374182410 2026-02-21T11:27:47.853 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:27:48.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182410 -lt 107374182410 2026-02-21T11:27:48.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:27:48.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:48.082 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:27:48.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:27:48.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:27:48.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:27:48.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:27:48.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-74328-1 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-74328-1 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:27:48.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-74328-1 td/test-erasure-eio/COPY 2026-02-21T11:27:48.872 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:27:48.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:27:48.875 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-21T11:27:48.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:27:48.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:27:48.876 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:27:49.160 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:27:49.169 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:27:49.464 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:27:49.474 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:27:49.586 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:27:49.587 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:27:49.587 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:27:49.588 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:27:49.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:27:49.589 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:27:49.590 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:27:49.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:49.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:27:49.591 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:27:49.592 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:49.592 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:27:49.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:27:49.594 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:27:49.609 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:27:49.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.610 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:27:49.612 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:27:49.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:27:49.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:27:49.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:27:49.614 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:27:49.615 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:27:49.615 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:27:49.615 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:27:49.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:27:49.616 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:27:49.617 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:27:49.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:49.617 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:27:49.618 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:27:49.618 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:27:49.619 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:27:49.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:27:49.620 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:27:49.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:27:49.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.621 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:27:49.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:27:49.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:27:49.623 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:27:49.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:27:49.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.625 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:27:49.626 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:27:49.627 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.650 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:49.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:27:49.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:27:49.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:27:49.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:27:49.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:27:49.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:27:49.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:27:49.682 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:27:49.682 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:27:49.682 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:27:49.682 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:49.682 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.683 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.683 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:27:49.683 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:27:49.683 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:27:49.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:27:49.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:27:49.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:27:49.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:27:49.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:27:49.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:27:49.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:27:49.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:27:49.909 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:27:49.909 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:49.909 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:49.910 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:49.910 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:49.910 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:49.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:49.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:27:49.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:27:49.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:27:49.929 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:27:50.035 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:27:50.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:27:51.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:27:51.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:27:51.049 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:27:51.050 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:27:51.050 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:51.050 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:51.050 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:27:51.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:27:51.050 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:27:51.098 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:27:51.103 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:27:51.105 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:27:49.663+0000 7fb0a990cec0 0 load: jerasure load: lrc 2026-02-21T11:27:51.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-21T11:27:51.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-21T11:27:51.106 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-21T11:27:51.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-21T11:27:51.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:27:51.107 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:27:51.107 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:27:51.107 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:51.111 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:51.112 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:51.113 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:51.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:51.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:51.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:51.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:51.114 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:27:51.115 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:51.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3ebbd2cf-cc8a-4aea-90a6-9626618cc1cf 2026-02-21T11:27:51.115 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3ebbd2cf-cc8a-4aea-90a6-9626618cc1cf' 2026-02-21T11:27:51.115 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 3ebbd2cf-cc8a-4aea-90a6-9626618cc1cf 2026-02-21T11:27:51.116 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:51.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC3lplpgKM+BxAANIfIq0vKzuhp3mcDHmCOsQ== 2026-02-21T11:27:51.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC3lplpgKM+BxAANIfIq0vKzuhp3mcDHmCOsQ=="}' 2026-02-21T11:27:51.123 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3ebbd2cf-cc8a-4aea-90a6-9626618cc1cf -i td/test-erasure-eio/0/new.json 2026-02-21T11:27:51.233 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:51.240 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:27:51.241 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC3lplpgKM+BxAANIfIq0vKzuhp3mcDHmCOsQ== --osd-uuid 3ebbd2cf-cc8a-4aea-90a6-9626618cc1cf 2026-02-21T11:27:51.257 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:51.251+0000 7f7e8a9ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:51.258 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:51.255+0000 7f7e8a9ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:51.259 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:51.255+0000 7f7e8a9ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:51.260 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:51.255+0000 7f7e8a9ada40 -1 bdev(0x55d11dc52400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:51.260 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:51.255+0000 7f7e8a9ada40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:27:51.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:27:51.965 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:51.966 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:27:51.966 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:27:51.967 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:52.081 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:27:52.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:27:52.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:52.082 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:52.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:52.105 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:52.146 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:52.139+0000 7fd7a9477a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:52.148 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:52.143+0000 7fd7a9477a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:52.149 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:52.143+0000 7fd7a9477a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:52.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:27:52.352 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:52.524 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:52.519+0000 7fd7a9477a40 -1 Falling back to public interface 2026-02-21T11:27:53.059 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:53.055+0000 7fd7a9477a40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:27:53.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:53.353 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:53.353 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:53.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:53.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:53.354 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:27:53.570 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:54.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:54.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:54.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:27:54.571 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:54.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:54.571 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:27:54.792 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1343992647,v1:127.0.0.1:6803/1343992647] [v2:127.0.0.1:6804/1343992647,v1:127.0.0.1:6805/1343992647] exists,up 3ebbd2cf-cc8a-4aea-90a6-9626618cc1cf 2026-02-21T11:27:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:54.795 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:54.796 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:54.797 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:54.798 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:54.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:54.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:27:54.799 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:54.799 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7912f8df-9f1d-4479-ad78-d674d7042ac4 2026-02-21T11:27:54.800 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 7912f8df-9f1d-4479-ad78-d674d7042ac4 2026-02-21T11:27:54.800 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7912f8df-9f1d-4479-ad78-d674d7042ac4' 2026-02-21T11:27:54.800 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:54.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC6lplp/YIaMBAAhFV1t6XegYmo0z2omcZvoA== 2026-02-21T11:27:54.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC6lplp/YIaMBAAhFV1t6XegYmo0z2omcZvoA=="}' 2026-02-21T11:27:54.808 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7912f8df-9f1d-4479-ad78-d674d7042ac4 -i td/test-erasure-eio/1/new.json 2026-02-21T11:27:55.026 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:55.037 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:27:55.038 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC6lplp/YIaMBAAhFV1t6XegYmo0z2omcZvoA== --osd-uuid 7912f8df-9f1d-4479-ad78-d674d7042ac4 2026-02-21T11:27:55.063 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.059+0000 7f295b92da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:55.065 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.059+0000 7f295b92da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:55.066 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.063+0000 7f295b92da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:55.067 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.063+0000 7f295b92da40 -1 bdev(0x565217f96400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:55.067 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.063+0000 7f295b92da40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:27:55.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:27:55.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:55.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:27:55.689 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:27:55.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:27:55.977 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:27:55.978 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:27:55.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:27:55.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:27:55.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:27:55.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:27:55.993 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.987+0000 7f018d190a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:55.994 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.991+0000 7f018d190a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:55.995 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:55.991+0000 7f018d190a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:56.190 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:27:56.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:27:56.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:56.191 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:56.348 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:56.343+0000 7f018d190a40 -1 Falling back to public interface 2026-02-21T11:27:56.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:57.019 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:57.015+0000 7f018d190a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:27:57.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:57.407 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:57.407 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:27:57.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:27:57.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:57.408 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:57.622 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:27:58.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:27:58.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:27:58.624 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:58.624 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:27:58.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:27:58.625 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:27:58.839 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/103491508,v1:127.0.0.1:6811/103491508] [v2:127.0.0.1:6812/103491508,v1:127.0.0.1:6813/103491508] exists,up 7912f8df-9f1d-4479-ad78-d674d7042ac4 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:27:58.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:27:58.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:27:58.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=55959d14-c9ce-41fa-8d98-deedee706861 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 55959d14-c9ce-41fa-8d98-deedee706861 2026-02-21T11:27:58.844 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 55959d14-c9ce-41fa-8d98-deedee706861' 2026-02-21T11:27:58.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:27:58.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+lplp7zgFMxAAfwqOL8hMinlxGDHBuzJRkA== 2026-02-21T11:27:58.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+lplp7zgFMxAAfwqOL8hMinlxGDHBuzJRkA=="}' 2026-02-21T11:27:58.857 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 55959d14-c9ce-41fa-8d98-deedee706861 -i td/test-erasure-eio/2/new.json 2026-02-21T11:27:59.083 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:27:59.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:27:59.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+lplp7zgFMxAAfwqOL8hMinlxGDHBuzJRkA== --osd-uuid 55959d14-c9ce-41fa-8d98-deedee706861 2026-02-21T11:27:59.110 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:59.107+0000 7f38b46c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:59.112 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:59.107+0000 7f38b46c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:59.113 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:59.107+0000 7f38b46c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:27:59.113 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:59.107+0000 7f38b46c6a40 -1 bdev(0x555aac294400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:27:59.114 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:27:59.107+0000 7f38b46c6a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:27:59.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:27:59.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:27:59.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:27:59.738 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:27:59.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:28:00.032 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:28:00.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:28:00.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:00.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:28:00.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:28:00.033 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:28:00.048 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:00.043+0000 7f9c743aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:00.049 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:00.043+0000 7f9c743aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:00.050 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:00.047+0000 7f9c743aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:00.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:28:00.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:00.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:28:00.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:00.265 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:00.265 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:00.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:00.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:00.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:00.266 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:00.470 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:00.543 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:00.539+0000 7f9c743aba40 -1 Falling back to public interface 2026-02-21T11:28:01.119 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:01.115+0000 7f9c743aba40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:28:01.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:01.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:01.472 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:01.473 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:01.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:01.473 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:01.685 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:02.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:02.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:02.686 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:28:02.686 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:28:02.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:02.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:02.901 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3550873386,v1:127.0.0.1:6819/3550873386] [v2:127.0.0.1:6820/3550873386,v1:127.0.0.1:6821/3550873386] exists,up 55959d14-c9ce-41fa-8d98-deedee706861 2026-02-21T11:28:02.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:02.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:02.905 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:02.906 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:02.907 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:02.908 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:28:02.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:28:02.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:28:02.909 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:28:02.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8bfc5527-cc6f-4f82-b401-c469ecbd1baf 2026-02-21T11:28:02.909 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 8bfc5527-cc6f-4f82-b401-c469ecbd1baf' 2026-02-21T11:28:02.910 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 8bfc5527-cc6f-4f82-b401-c469ecbd1baf 2026-02-21T11:28:02.910 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:28:02.917 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDClplp/uyeNhAAzjii5m5zKVE1Io2zZ3o6Kw== 2026-02-21T11:28:02.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDClplp/uyeNhAAzjii5m5zKVE1Io2zZ3o6Kw=="}' 2026-02-21T11:28:02.918 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8bfc5527-cc6f-4f82-b401-c469ecbd1baf -i td/test-erasure-eio/3/new.json 2026-02-21T11:28:03.140 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:28:03.150 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:28:03.151 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDClplp/uyeNhAAzjii5m5zKVE1Io2zZ3o6Kw== --osd-uuid 8bfc5527-cc6f-4f82-b401-c469ecbd1baf 2026-02-21T11:28:03.176 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:03.171+0000 7ff4ba1e8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:03.177 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:03.171+0000 7ff4ba1e8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:03.178 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:03.175+0000 7ff4ba1e8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:03.179 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:03.175+0000 7ff4ba1e8a40 -1 bdev(0x561e8a4e4400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:28:03.179 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:03.175+0000 7ff4ba1e8a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:28:03.792 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:28:03.793 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:28:03.793 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:28:03.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:28:03.794 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:28:04.099 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:28:04.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:28:04.099 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:04.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:28:04.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:28:04.100 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:28:04.123 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:04.119+0000 7f4d9d2c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:04.124 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:04.119+0000 7f4d9d2c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:04.125 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:04.119+0000 7f4d9d2c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:04.308 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:04.309 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:04.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:04.310 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:28:04.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:04.571 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:04.567+0000 7f4d9d2c2a40 -1 Falling back to public interface 2026-02-21T11:28:05.157 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:05.151+0000 7f4d9d2c2a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:28:05.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:05.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:05.524 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:05.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:05.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:05.525 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:28:05.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:06.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:06.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:06.737 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:28:06.738 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:28:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:06.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:28:06.950 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 24 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/953560572,v1:127.0.0.1:6827/953560572] [v2:127.0.0.1:6828/953560572,v1:127.0.0.1:6829/953560572] exists,up 8bfc5527-cc6f-4f82-b401-c469ecbd1baf 2026-02-21T11:28:06.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:06.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:06.952 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:06.953 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:28:06.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:28:06.953 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:28:07.006 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:28:07.012 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:28:07.013 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:27:52.523+0000 7fd7a9477a40 0 load: jerasure load: lrc 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:28:07.014 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:28:07.286 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:28:07.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:28:07.637 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:28:07.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:28:08.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:28:08.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:28:08.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:28:08.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:28:08.651 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:28:08.652 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:28:08.652 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:28:08.652 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:28:08.652 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:28:08.652 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:28:08.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:28:08.759 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:28:08.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:28:08.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:28:08.760 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:28:08.760 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:08.972 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:28:09.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-21T11:28:09.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-21T11:28:09.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-21T11:28:09.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:09.052 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:28:09.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607557 2026-02-21T11:28:09.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607557 2026-02-21T11:28:09.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607557' 2026-02-21T11:28:09.131 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:09.131 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:28:09.212 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444036 2026-02-21T11:28:09.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444036 2026-02-21T11:28:09.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036' 2026-02-21T11:28:09.213 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:09.213 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:28:09.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=103079215108 2026-02-21T11:28:09.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 103079215108 2026-02-21T11:28:09.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036 3-103079215108' 2026-02-21T11:28:09.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:09.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-21T11:28:09.295 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:09.295 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:28:09.296 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-21T11:28:09.296 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:09.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-21T11:28:09.298 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803781 2026-02-21T11:28:09.298 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-21T11:28:09.298 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:28:09.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-21T11:28:09.512 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:28:10.513 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:28:10.513 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:28:10.726 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-21T11:28:10.727 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:10.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607557 2026-02-21T11:28:10.727 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:10.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:28:10.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607557 2026-02-21T11:28:10.729 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:10.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607557 2026-02-21T11:28:10.731 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607557 2026-02-21T11:28:10.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-21T11:28:10.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:28:10.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-21T11:28:10.948 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:10.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444036 2026-02-21T11:28:10.949 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:10.950 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:28:10.951 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444036 2026-02-21T11:28:10.951 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:10.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444036 2026-02-21T11:28:10.952 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-21T11:28:10.953 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 73014444036 2026-02-21T11:28:10.953 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:28:11.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-21T11:28:11.170 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:11.171 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-103079215108 2026-02-21T11:28:11.171 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:11.172 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:28:11.173 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-103079215108 2026-02-21T11:28:11.173 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:11.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=103079215108 2026-02-21T11:28:11.174 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 103079215108' 2026-02-21T11:28:11.175 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 103079215108 2026-02-21T11:28:11.175 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:28:11.394 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 103079215108 -lt 103079215108 2026-02-21T11:28:11.395 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:28:11.395 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:11.395 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:11.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:28:11.679 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:28:11.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:28:11.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:28:11.680 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:28:11.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:28:11.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:28:11.681 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:28:11.889 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:28:11.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:28:11.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:11.890 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:12.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:28:12.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:28:12.179 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74328-0 2026-02-21T11:28:12.180 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74328-0 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74328-0 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:28:12.181 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74328-0 td/test-erasure-eio/ORIGINAL 2026-02-21T11:28:12.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74328-0 td/test-erasure-eio 0 2026-02-21T11:28:12.198 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-74328-0 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:28:12.199 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-74328-0 2026-02-21T11:28:12.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:28:12.200 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-0 2026-02-21T11:28:12.200 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-0 2026-02-21T11:28:12.200 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:28:12.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:28:12.418 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:12.418 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:28:12.418 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:28:12.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-21T11:28:12.418 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:28:12.419 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:12.420 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:28:12.421 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:28:12.481 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:28:12.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:28:12.482 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:12.483 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.3.asok 2026-02-21T11:28:12.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:28:12.484 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-74328-0 0 2026-02-21T11:28:12.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74328-0 2026-02-21T11:28:12.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:28:12.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:28:12.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74328-0 2026-02-21T11:28:12.543 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:28:12.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:28:12.544 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74328-0 td/test-erasure-eio/COPY 2026-02-21T11:28:12.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:28:12.564 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:28:12.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-21T11:28:12.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-74328-0 2026-02-21T11:28:12.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:28:12.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-0 2026-02-21T11:28:12.567 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-0 2026-02-21T11:28:12.567 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-21T11:28:12.782 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-21T11:28:12.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:28:12.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:28:12.783 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:28:12.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:28:12.783 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:28:12.784 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:28:13.091 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:28:13.092 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-21T11:28:13.334 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already out. 2026-02-21T11:28:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-74328-0 2026-02-21T11:28:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:28:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-0 2026-02-21T11:28:13.345 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-21T11:28:13.345 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-0 2026-02-21T11:28:13.346 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:28:13.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:28:13.561 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:13.561 INFO:tasks.workunit.client.0.trial166.stderr:0' 2026-02-21T11:28:13.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-21T11:28:13.562 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-21T11:28:13.827 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already in. 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:13.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:13.842 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:28:13.843 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:28:13.844 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:28:13.845 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:28:13.863 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:13.859+0000 7f5fbabf0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:13.865 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:13.859+0000 7f5fbabf0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:13.866 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:13.863+0000 7f5fbabf0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:14.051 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:28:14.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:14.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:28:14.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:14.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:14.052 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:14.053 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:14.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:14.053 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:14.054 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:14.273 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:14.303 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:14.299+0000 7f5fbabf0a40 -1 Falling back to public interface 2026-02-21T11:28:14.873 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:14.867+0000 7f5fbabf0a40 -1 osd.2 31 log_to_monitors true 2026-02-21T11:28:15.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:15.274 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:15.274 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:15.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:15.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:15.275 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:15.489 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 40 up_thru 17 down_at 32 last_clean_interval [17,31) [v2:127.0.0.1:6818/147377857,v1:127.0.0.1:6819/147377857] [v2:127.0.0.1:6820/147377857,v1:127.0.0.1:6821/147377857] exists,up 55959d14-c9ce-41fa-8d98-deedee706861 2026-02-21T11:28:15.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:15.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:15.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:15.491 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:28:15.492 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:28:15.597 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:28:15.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:28:15.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:28:15.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:28:15.598 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:28:15.598 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:15.818 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:28:15.897 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803785 2026-02-21T11:28:15.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803785 2026-02-21T11:28:15.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785' 2026-02-21T11:28:15.898 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:15.898 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:28:15.978 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607560 2026-02-21T11:28:15.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607560 2026-02-21T11:28:15.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-51539607560' 2026-02-21T11:28:15.979 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:15.979 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:28:16.057 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=171798691843 2026-02-21T11:28:16.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 171798691843 2026-02-21T11:28:16.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-51539607560 2-171798691843' 2026-02-21T11:28:16.058 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:16.058 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:28:16.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=103079215111 2026-02-21T11:28:16.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 103079215111 2026-02-21T11:28:16.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-51539607560 2-171798691843 3-103079215111' 2026-02-21T11:28:16.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:16.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803785 2026-02-21T11:28:16.137 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:16.138 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:28:16.139 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803785 2026-02-21T11:28:16.139 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:16.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803785 2026-02-21T11:28:16.140 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803785 2026-02-21T11:28:16.140 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-21T11:28:16.141 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:28:16.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-21T11:28:16.359 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:16.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607560 2026-02-21T11:28:16.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:16.361 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:28:16.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607560 2026-02-21T11:28:16.361 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:16.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607560 2026-02-21T11:28:16.363 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-21T11:28:16.363 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 51539607560 2026-02-21T11:28:16.363 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:28:16.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-21T11:28:16.581 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:16.581 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-171798691843 2026-02-21T11:28:16.582 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:16.583 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:28:16.583 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-171798691843 2026-02-21T11:28:16.584 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:16.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=171798691843 2026-02-21T11:28:16.585 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 171798691843' 2026-02-21T11:28:16.585 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 171798691843 2026-02-21T11:28:16.586 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:28:16.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 171798691843 -lt 171798691843 2026-02-21T11:28:16.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:16.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-103079215111 2026-02-21T11:28:16.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:16.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:28:16.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-103079215111 2026-02-21T11:28:16.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:16.818 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=103079215111 2026-02-21T11:28:16.819 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 103079215111 2026-02-21T11:28:16.819 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 103079215111' 2026-02-21T11:28:16.819 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:28:17.031 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 103079215111 -lt 103079215111 2026-02-21T11:28:17.031 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:28:17.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:17.032 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:28:17.309 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:28:17.310 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:28:17.514 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:28:17.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:28:17.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:17.515 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:17.803 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:28:17.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:28:17.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:28:17.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:28:17.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:28:17.804 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:28:18.056 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:28:18.065 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:28:18.360 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:28:18.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:28:18.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:28:18.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:28:18.371 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:28:18.371 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:28:18.372 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:28:18.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:28:18.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:28:18.372 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:28:18.683 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:28:18.683 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:28:18.684 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:28:18.685 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:28:18.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:28:18.687 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:28:18.687 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:28:18.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:18.688 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:28:18.689 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:28:18.689 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:18.690 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:28:18.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:28:18.691 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:28:18.707 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:28:18.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:18.708 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:18.708 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:28:18.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:28:18.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:28:18.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-21T11:28:18.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-21T11:28:18.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-21T11:28:18.710 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:28:18.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:28:18.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:28:18.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:28:18.714 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:28:18.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:28:18.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:28:18.716 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:28:18.716 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:28:18.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:18.717 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:28:18.718 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:28:18.718 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:18.719 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:28:18.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:28:18.720 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:28:18.721 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:28:18.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:18.722 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:18.722 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:28:18.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:28:18.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:28:18.723 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-21T11:28:18.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T11:28:18.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:18.724 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:18.724 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74328 2026-02-21T11:28:18.726 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-21T11:28:18.728 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:18.750 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:18.751 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T11:28:18.777 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T11:28:18.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T11:28:18.778 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:28:18.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:28:18.779 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T11:28:18.780 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T11:28:18.780 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:28:18.780 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:28:18.780 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:28:18.780 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:18.780 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:18.781 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:18.781 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:28:18.781 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:28:18.782 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get fsid 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:28:18.835 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:28:18.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:18.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:18.836 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:18.836 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:28:18.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T11:28:18.836 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74328/ceph-mon.a.asok config get mon_host 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-21T11:28:18.891 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T11:28:19.005 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T11:28:19.005 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:19.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:19.006 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:19.006 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:19.006 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:19.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:19.006 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:28:19.007 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T11:28:19.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-21T11:28:19.024 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-21T11:28:19.141 INFO:tasks.workunit.client.0.trial166.stderr:pool 'rbd' already exists 2026-02-21T11:28:19.149 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:28:20.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-21T11:28:20.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T11:28:20.152 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T11:28:20.152 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:20.153 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:20.153 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:20.153 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-mon.a.asok 2026-02-21T11:28:20.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-21T11:28:20.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-mon.a.asok log flush 2026-02-21T11:28:20.202 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:28:20.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-21T11:28:20.209 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:28:18.763+0000 7f3dc9b4cec0 0 load: jerasure load: lrc 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-21T11:28:20.211 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-21T11:28:20.212 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-21T11:28:20.215 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:20.216 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:20.217 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:28:20.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:28:20.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-21T11:28:20.219 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:28:20.219 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6933be1b-b567-4e89-8f0a-34c7b0564084 2026-02-21T11:28:20.220 INFO:tasks.workunit.client.0.trial166.stdout:add osd0 6933be1b-b567-4e89-8f0a-34c7b0564084 2026-02-21T11:28:20.220 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6933be1b-b567-4e89-8f0a-34c7b0564084' 2026-02-21T11:28:20.220 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:28:20.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDUlplpDKOIDRAAbgc6lEy7Z0gEPvzpAOj9rA== 2026-02-21T11:28:20.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDUlplpDKOIDRAAbgc6lEy7Z0gEPvzpAOj9rA=="}' 2026-02-21T11:28:20.228 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6933be1b-b567-4e89-8f0a-34c7b0564084 -i td/test-erasure-eio/0/new.json 2026-02-21T11:28:20.337 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:20.344 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-21T11:28:20.346 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUlplpDKOIDRAAbgc6lEy7Z0gEPvzpAOj9rA== --osd-uuid 6933be1b-b567-4e89-8f0a-34c7b0564084 2026-02-21T11:28:20.361 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:20.355+0000 7f62f3c1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:20.362 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:20.359+0000 7f62f3c1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:20.363 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:20.359+0000 7f62f3c1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:20.364 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:20.359+0000 7f62f3c1ba40 -1 bdev(0x558032c1c400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-21T11:28:20.364 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:20.359+0000 7f62f3c1ba40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-21T11:28:21.017 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-21T11:28:21.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:28:21.018 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T11:28:21.019 INFO:tasks.workunit.client.0.trial166.stdout:adding osd0 key to auth repository 2026-02-21T11:28:21.019 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:28:21.135 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T11:28:21.136 INFO:tasks.workunit.client.0.trial166.stdout:start osd.0 2026-02-21T11:28:21.136 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:21.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:28:21.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:28:21.137 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:28:21.151 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:21.147+0000 7f58e827ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:21.152 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:21.147+0000 7f58e827ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:21.153 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:21.147+0000 7f58e827ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:21.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:21.282 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:21.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:21.283 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:28:21.391 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:21.592 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:21.587+0000 7f58e827ea40 -1 Falling back to public interface 2026-02-21T11:28:22.248 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:22.243+0000 7f58e827ea40 -1 osd.0 0 log_to_monitors true 2026-02-21T11:28:22.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:22.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:22.392 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:22.393 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:22.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:22.393 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:28:22.610 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:23.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:23.611 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:23.612 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:28:23.612 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:28:23.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:23.613 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T11:28:23.823 INFO:tasks.workunit.client.0.trial166.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1214520578,v1:127.0.0.1:6803/1214520578] [v2:127.0.0.1:6804/1214520578,v1:127.0.0.1:6805/1214520578] exists,up 6933be1b-b567-4e89-8f0a-34c7b0564084 2026-02-21T11:28:23.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:23.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:23.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:23.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:28:23.827 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-21T11:28:23.828 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:23.829 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-21T11:28:23.830 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:28:23.831 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1f8b3654-081d-40b2-93f2-9aa6e83eed12 2026-02-21T11:28:23.831 INFO:tasks.workunit.client.0.trial166.stdout:add osd1 1f8b3654-081d-40b2-93f2-9aa6e83eed12 2026-02-21T11:28:23.832 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1f8b3654-081d-40b2-93f2-9aa6e83eed12' 2026-02-21T11:28:23.832 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:28:23.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDXlplpmJcMMhAA7TZeCPlDuRsmAWuL3dmrJA== 2026-02-21T11:28:23.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDXlplpmJcMMhAA7TZeCPlDuRsmAWuL3dmrJA=="}' 2026-02-21T11:28:23.841 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1f8b3654-081d-40b2-93f2-9aa6e83eed12 -i td/test-erasure-eio/1/new.json 2026-02-21T11:28:24.066 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:24.077 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-21T11:28:24.078 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDXlplpmJcMMhAA7TZeCPlDuRsmAWuL3dmrJA== --osd-uuid 1f8b3654-081d-40b2-93f2-9aa6e83eed12 2026-02-21T11:28:24.095 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:24.091+0000 7faec6534a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:24.097 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:24.091+0000 7faec6534a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:24.098 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:24.091+0000 7faec6534a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:24.098 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:24.095+0000 7faec6534a40 -1 bdev(0x55d3e8016400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-21T11:28:24.098 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:24.095+0000 7faec6534a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-21T11:28:24.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-21T11:28:24.733 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:28:24.734 INFO:tasks.workunit.client.0.trial166.stdout:adding osd1 key to auth repository 2026-02-21T11:28:24.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T11:28:24.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:28:25.026 INFO:tasks.workunit.client.0.trial166.stdout:start osd.1 2026-02-21T11:28:25.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T11:28:25.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:25.026 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:28:25.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:28:25.027 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:28:25.050 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:25.047+0000 7fe7a8777a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:25.051 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:25.047+0000 7fe7a8777a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:25.052 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:25.047+0000 7fe7a8777a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:25.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:28:25.443 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:25.439+0000 7fe7a8777a40 -1 Falling back to public interface 2026-02-21T11:28:25.456 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:26.057 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:26.051+0000 7fe7a8777a40 -1 osd.1 0 log_to_monitors true 2026-02-21T11:28:26.457 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:26.458 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:26.458 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:26.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:26.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:26.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T11:28:26.664 INFO:tasks.workunit.client.0.trial166.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3495891237,v1:127.0.0.1:6811/3495891237] [v2:127.0.0.1:6812/3495891237,v1:127.0.0.1:6813/3495891237] exists,up 1f8b3654-081d-40b2-93f2-9aa6e83eed12 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:26.666 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:26.667 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:28:26.669 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:28:26.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:28:26.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d617fe24-a86d-4337-bc41-421972e188fc 2026-02-21T11:28:26.670 INFO:tasks.workunit.client.0.trial166.stdout:add osd2 d617fe24-a86d-4337-bc41-421972e188fc 2026-02-21T11:28:26.670 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d617fe24-a86d-4337-bc41-421972e188fc' 2026-02-21T11:28:26.670 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:28:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDalplpfSmHKBAALjRvRUvBQwwTXmawgiWiDg== 2026-02-21T11:28:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDalplpfSmHKBAALjRvRUvBQwwTXmawgiWiDg=="}' 2026-02-21T11:28:26.681 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d617fe24-a86d-4337-bc41-421972e188fc -i td/test-erasure-eio/2/new.json 2026-02-21T11:28:26.900 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:28:26.910 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-21T11:28:26.911 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDalplpfSmHKBAALjRvRUvBQwwTXmawgiWiDg== --osd-uuid d617fe24-a86d-4337-bc41-421972e188fc 2026-02-21T11:28:26.935 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:26.931+0000 7f8e5a02da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:26.937 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:26.931+0000 7f8e5a02da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:26.938 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:26.935+0000 7f8e5a02da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:26.938 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:26.935+0000 7f8e5a02da40 -1 bdev(0x560e03e82400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-21T11:28:26.939 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:26.935+0000 7f8e5a02da40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-21T11:28:27.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-21T11:28:27.566 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:28:27.566 INFO:tasks.workunit.client.0.trial166.stdout:adding osd2 key to auth repository 2026-02-21T11:28:27.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T11:28:27.567 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:28:27.863 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:28:27.864 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T11:28:27.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:27.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:28:27.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:28:27.865 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:28:27.887 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:27.883+0000 7fdcf0602a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:27.888 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:27.883+0000 7fdcf0602a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:27.889 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:27.883+0000 7fdcf0602a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:28.074 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:28.075 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:28.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:28.076 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:28.291 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:28.311 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:28.307+0000 7fdcf0602a40 -1 Falling back to public interface 2026-02-21T11:28:28.904 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:28.899+0000 7fdcf0602a40 -1 osd.2 0 log_to_monitors true 2026-02-21T11:28:29.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:29.293 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:29.294 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:29.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:29.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:29.294 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:29.517 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:30.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:30.518 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:30.519 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:28:30.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:28:30.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:30.519 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:30.733 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/711795657,v1:127.0.0.1:6819/711795657] [v2:127.0.0.1:6820/711795657,v1:127.0.0.1:6821/711795657] exists,up d617fe24-a86d-4337-bc41-421972e188fc 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-21T11:28:30.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:30.735 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T11:28:30.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-21T11:28:30.737 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T11:28:30.738 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=97560e5a-7921-40ca-84a6-b1acd61eadcb 2026-02-21T11:28:30.738 INFO:tasks.workunit.client.0.trial166.stdout:add osd3 97560e5a-7921-40ca-84a6-b1acd61eadcb 2026-02-21T11:28:30.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 97560e5a-7921-40ca-84a6-b1acd61eadcb' 2026-02-21T11:28:30.739 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T11:28:30.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDelplpAy3FLBAA8Xn8Z6D/lbDI/JxfQIxc1A== 2026-02-21T11:28:30.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDelplpAy3FLBAA8Xn8Z6D/lbDI/JxfQIxc1A=="}' 2026-02-21T11:28:30.753 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 97560e5a-7921-40ca-84a6-b1acd61eadcb -i td/test-erasure-eio/3/new.json 2026-02-21T11:28:30.976 INFO:tasks.workunit.client.0.trial166.stdout:3 2026-02-21T11:28:30.986 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-21T11:28:30.987 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDelplpAy3FLBAA8Xn8Z6D/lbDI/JxfQIxc1A== --osd-uuid 97560e5a-7921-40ca-84a6-b1acd61eadcb 2026-02-21T11:28:31.012 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.006+0000 7f52aeda3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:31.013 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.006+0000 7f52aeda3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:31.015 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.010+0000 7f52aeda3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:31.015 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.010+0000 7f52aeda3a40 -1 bdev(0x562fa420a400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-21T11:28:31.015 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.010+0000 7f52aeda3a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-21T11:28:31.645 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-21T11:28:31.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T11:28:31.646 INFO:tasks.workunit.client.0.trial166.stdout:adding osd3 key to auth repository 2026-02-21T11:28:31.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T11:28:31.647 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T11:28:31.941 INFO:tasks.workunit.client.0.trial166.stdout:start osd.3 2026-02-21T11:28:31.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T11:28:31.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:31.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T11:28:31.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T11:28:31.942 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T11:28:31.965 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.958+0000 7f0b9c206a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:31.966 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.962+0000 7f0b9c206a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:31.968 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:31.962+0000 7f0b9c206a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T11:28:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T11:28:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:32.152 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:32.153 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:32.153 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:28:32.359 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:32.354+0000 7f0b9c206a40 -1 Falling back to public interface 2026-02-21T11:28:32.374 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:33.028 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:33.022+0000 7f0b9c206a40 -1 osd.3 0 log_to_monitors true 2026-02-21T11:28:33.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:33.375 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:33.376 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:33.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:33.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:33.376 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:28:33.593 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:34.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:34.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:34.595 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T11:28:34.596 INFO:tasks.workunit.client.0.trial166.stdout:2 2026-02-21T11:28:34.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:34.596 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T11:28:34.809 INFO:tasks.workunit.client.0.trial166.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/495163838,v1:127.0.0.1:6827/495163838] [v2:127.0.0.1:6828/495163838,v1:127.0.0.1:6829/495163838] exists,up 97560e5a-7921-40ca-84a6-b1acd61eadcb 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T11:28:34.810 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:34.811 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:34.811 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:34.811 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.0.asok 2026-02-21T11:28:34.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-21T11:28:34.811 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.0.asok log flush 2026-02-21T11:28:34.874 INFO:tasks.workunit.client.0.trial166.stdout:{} 2026-02-21T11:28:34.880 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-21T11:28:34.882 INFO:tasks.workunit.client.0.trial166.stdout:2026-02-21T11:28:21.591+0000 7f58e827ea40 0 load: jerasure load: lrc 2026-02-21T11:28:34.882 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-21T11:28:34.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-21T11:28:35.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-21T11:28:35.161 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-21T11:28:35.550 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' already exists 2026-02-21T11:28:35.560 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T11:28:36.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-21T11:28:36.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:28:36.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:28:36.561 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:28:36.562 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:28:36.563 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:28:36.563 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:28:36.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:28:36.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:28:36.563 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:28:36.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:28:36.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:28:36.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:28:36.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:28:36.665 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:28:36.665 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:28:36.883 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:28:36.883 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:36.883 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:28:36.884 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:28:36.884 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:28:36.884 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:36.884 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:28:36.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-21T11:28:36.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-21T11:28:36.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-21T11:28:36.964 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:36.964 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:28:37.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-21T11:28:37.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-21T11:28:37.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-21T11:28:37.046 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:37.046 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:28:37.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378628 2026-02-21T11:28:37.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378628 2026-02-21T11:28:37.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378628' 2026-02-21T11:28:37.128 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:37.128 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:28:37.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149699 2026-02-21T11:28:37.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149699 2026-02-21T11:28:37.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378628 3-111669149699' 2026-02-21T11:28:37.206 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:37.207 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-21T11:28:37.207 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:37.208 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:28:37.209 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-21T11:28:37.209 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:37.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-21T11:28:37.211 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803781 2026-02-21T11:28:37.211 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-21T11:28:37.211 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:28:37.426 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-21T11:28:37.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:37.427 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-21T11:28:37.427 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:37.429 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:28:37.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-21T11:28:37.430 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-21T11:28:37.431 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574852 2026-02-21T11:28:37.431 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-21T11:28:37.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:28:37.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574853 -lt 55834574852 2026-02-21T11:28:37.646 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:37.647 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378628 2026-02-21T11:28:37.647 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:37.648 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:28:37.648 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378628 2026-02-21T11:28:37.649 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:37.650 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378628 2026-02-21T11:28:37.651 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-21T11:28:37.651 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 81604378628 2026-02-21T11:28:37.651 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:28:37.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-21T11:28:37.867 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:37.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149699 2026-02-21T11:28:37.867 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:37.868 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:28:37.869 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149699 2026-02-21T11:28:37.869 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:37.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149699 2026-02-21T11:28:37.871 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149699 2026-02-21T11:28:37.871 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-21T11:28:37.871 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:28:38.080 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-21T11:28:38.080 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:28:38.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:38.081 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:38.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:28:38.358 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:28:38.358 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:28:38.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:28:38.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:28:38.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:28:38.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:28:38.359 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:28:38.565 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:28:38.565 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:28:38.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:38.566 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74328-1 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74328-1 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74328-1 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-21T11:28:38.852 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:38.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-21T11:28:38.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-21T11:28:38.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-21T11:28:38.853 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74328-1 td/test-erasure-eio/ORIGINAL 2026-02-21T11:28:38.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74328-1 td/test-erasure-eio 1 2026-02-21T11:28:38.873 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-74328-1 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-21T11:28:38.874 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-21T11:28:38.875 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-74328-1 2026-02-21T11:28:38.875 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:28:38.875 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-1 2026-02-21T11:28:38.875 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-1 2026-02-21T11:28:38.875 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:28:39.092 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:28:39.092 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:39.093 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:28:39.093 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:28:39.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-21T11:28:39.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-21T11:28:39.093 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-21T11:28:39.094 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-21T11:28:39.094 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-21T11:28:39.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-21T11:28:39.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-21T11:28:39.096 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:28:39.097 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74328/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-21T11:28:39.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T11:28:39.155 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-21T11:28:39.156 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-21T11:28:39.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-21T11:28:39.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T11:28:39.156 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T11:28:39.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T11:28:39.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:39.157 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:39.157 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74328/ceph-osd.1.asok 2026-02-21T11:28:39.157 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-21T11:28:39.158 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74328/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-74328-1 1 2026-02-21T11:28:39.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74328-1 2026-02-21T11:28:39.217 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-21T11:28:39.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-21T11:28:39.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74328-1 2026-02-21T11:28:39.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-21T11:28:39.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-21T11:28:39.218 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74328-1 td/test-erasure-eio/COPY 2026-02-21T11:28:39.237 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-21T11:28:39.238 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-21T11:28:39.239 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-21T11:28:39.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-74328-1 2026-02-21T11:28:39.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:28:39.241 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-1 2026-02-21T11:28:39.242 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-1 2026-02-21T11:28:39.242 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:28:39.458 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr:2' 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-21T11:28:39.459 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:28:39.460 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:28:39.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:28:39.460 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:28:39.461 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:28:39.767 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:28:39.768 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-21T11:28:40.034 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already out. 2026-02-21T11:28:40.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-74328-1 2026-02-21T11:28:40.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-21T11:28:40.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74328-1 2026-02-21T11:28:40.045 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-21T11:28:40.046 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74328-1 2026-02-21T11:28:40.046 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T11:28:40.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-21T11:28:40.262 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:40.262 INFO:tasks.workunit.client.0.trial166.stderr:0' 2026-02-21T11:28:40.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-21T11:28:40.263 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-21T11:28:40.510 INFO:tasks.workunit.client.0.trial166.stderr:osd.2 is already in. 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T11:28:40.521 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:40.522 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T11:28:40.523 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-21T11:28:40.524 INFO:tasks.workunit.client.0.trial166.stdout:start osd.2 2026-02-21T11:28:40.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T11:28:40.524 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d13dfb32-4938-4c67-8632-6fee49a96779 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74328/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T11:28:40.525 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-21T11:28:40.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T11:28:40.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T11:28:40.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T11:28:40.526 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T11:28:40.548 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:40.542+0000 7f2d6e17da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:40.549 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:40.542+0000 7f2d6e17da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:40.550 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:40.546+0000 7f2d6e17da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T11:28:40.734 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T11:28:40.735 INFO:tasks.workunit.client.0.trial166.stdout:0 2026-02-21T11:28:40.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:40.736 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:40.949 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T11:28:40.991 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:40.986+0000 7f2d6e17da40 -1 Falling back to public interface 2026-02-21T11:28:41.578 INFO:tasks.workunit.client.0.trial166.stderr:2026-02-21T11:28:41.574+0000 7f2d6e17da40 -1 osd.2 33 log_to_monitors true 2026-02-21T11:28:41.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T11:28:41.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T11:28:41.951 INFO:tasks.workunit.client.0.trial166.stdout:1 2026-02-21T11:28:41.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T11:28:41.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T11:28:41.951 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T11:28:42.167 INFO:tasks.workunit.client.0.trial166.stdout:osd.2 up in weight 1 up_from 42 up_thru 19 down_at 34 last_clean_interval [19,33) [v2:127.0.0.1:6818/3599959955,v1:127.0.0.1:6819/3599959955] [v2:127.0.0.1:6820/3599959955,v1:127.0.0.1:6821/3599959955] exists,up d617fe24-a86d-4337-bc41-421972e188fc 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T11:28:42.168 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T11:28:42.169 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T11:28:42.169 INFO:tasks.workunit.client.0.trial166.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T11:28:42.169 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T11:28:42.169 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T11:28:42.169 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T11:28:42.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T11:28:42.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T11:28:42.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T11:28:42.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T11:28:42.281 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-21T11:28:42.281 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-21T11:28:42.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-21T11:28:42.492 INFO:tasks.workunit.client.0.trial166.stderr:1 2026-02-21T11:28:42.492 INFO:tasks.workunit.client.0.trial166.stderr:2 2026-02-21T11:28:42.492 INFO:tasks.workunit.client.0.trial166.stderr:3' 2026-02-21T11:28:42.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-21T11:28:42.492 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:42.493 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-21T11:28:42.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803784 2026-02-21T11:28:42.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803784 2026-02-21T11:28:42.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784' 2026-02-21T11:28:42.574 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:42.574 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-21T11:28:42.652 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574856 2026-02-21T11:28:42.652 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574856 2026-02-21T11:28:42.652 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574856' 2026-02-21T11:28:42.652 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:42.653 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-21T11:28:42.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626435 2026-02-21T11:28:42.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626435 2026-02-21T11:28:42.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-180388626435' 2026-02-21T11:28:42.731 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-21T11:28:42.731 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-21T11:28:42.812 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149702 2026-02-21T11:28:42.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149702 2026-02-21T11:28:42.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-180388626435 3-111669149702' 2026-02-21T11:28:42.813 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:42.813 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803784 2026-02-21T11:28:42.813 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:42.814 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-21T11:28:42.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803784 2026-02-21T11:28:42.815 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:42.816 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803784 2026-02-21T11:28:42.817 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.0 seq 25769803784 2026-02-21T11:28:42.817 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-21T11:28:42.817 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:28:43.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-21T11:28:43.039 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-21T11:28:44.040 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T11:28:44.040 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T11:28:44.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-21T11:28:44.258 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:44.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574856 2026-02-21T11:28:44.259 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:44.260 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-21T11:28:44.260 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574856 2026-02-21T11:28:44.261 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:44.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574856 2026-02-21T11:28:44.262 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-21T11:28:44.263 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.1 seq 55834574856 2026-02-21T11:28:44.263 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T11:28:44.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-21T11:28:44.475 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:44.475 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-180388626435 2026-02-21T11:28:44.475 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:44.477 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-21T11:28:44.478 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-180388626435 2026-02-21T11:28:44.478 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:44.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626435 2026-02-21T11:28:44.479 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-21T11:28:44.479 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.2 seq 180388626435 2026-02-21T11:28:44.479 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T11:28:44.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-21T11:28:44.711 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-21T11:28:44.711 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149702 2026-02-21T11:28:44.711 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-21T11:28:44.713 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-21T11:28:44.713 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149702 2026-02-21T11:28:44.714 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-21T11:28:44.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149702 2026-02-21T11:28:44.715 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-21T11:28:44.715 INFO:tasks.workunit.client.0.trial166.stdout:waiting osd.3 seq 111669149702 2026-02-21T11:28:44.715 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T11:28:44.938 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-21T11:28:44.939 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T11:28:44.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:44.940 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T11:28:45.229 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T11:28:45.230 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T11:28:45.443 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-21T11:28:45.443 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T11:28:45.444 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T11:28:45.444 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T11:28:45.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-21T11:28:45.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T11:28:45.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T11:28:45.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-21T11:28:45.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-21T11:28:45.739 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-21T11:28:45.993 INFO:tasks.workunit.client.0.trial166.stderr:pool 'pool-jerasure' does not exist 2026-02-21T11:28:46.003 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-21T11:28:46.277 INFO:tasks.workunit.client.0.trial166.stderr:erasure-code-profile myprofile does not exist 2026-02-21T11:28:46.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-21T11:28:46.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:28:46.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T11:28:46.287 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:28:46.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:28:46.287 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:28:46.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:28:46.288 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:28:46.289 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:28:46.398 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:28:46.398 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:28:46.399 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:28:46.400 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:28:46.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:28:46.401 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:28:46.402 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:28:46.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:46.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:28:46.403 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:28:46.404 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:46.405 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:28:46.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T11:28:46.406 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:28:46.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:28:46.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:46.422 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:46.423 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:28:46.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:28:46.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:28:46.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-21T11:28:46.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/test-erasure-eio 0 2026-02-21T11:28:46.424 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T11:28:46.425 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T11:28:46.427 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T11:28:46.427 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T11:28:46.428 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T11:28:46.429 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T11:28:46.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T11:28:46.430 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T11:28:46.431 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T11:28:46.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:46.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T11:28:46.432 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T11:28:46.433 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T11:28:46.434 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:28:46.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T11:28:46.435 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-21T11:28:46.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T11:28:46.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T11:28:46.436 INFO:tasks.workunit.client.0.trial166.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74328 2026-02-21T11:28:46.436 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74328 2026-02-21T11:28:46.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T11:28:46.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T11:28:46.437 INFO:tasks.workunit.client.0.trial166.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-21T11:28:46.439 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T11:28:46.439 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T11:28:46.489 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-21T11:28:46.489 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-21T11:28:46.851 DEBUG:teuthology.parallel:result is None 2026-02-21T11:28:46.851 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T11:28:46.858 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T11:28:46.858 DEBUG:teuthology.orchestra.run.trial166:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T11:28:46.905 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T11:28:46.906 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T11:28:46.920 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T11:28:46.920 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T11:28:47.060 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T11:28:47.072 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T11:28:47.072 DEBUG:teuthology.orchestra.run.trial166:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T11:28:47.256 INFO:teuthology.orchestra.run.trial166.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T11:28:47.257 INFO:teuthology.orchestra.run.trial166.stdout:============================================================================== 2026-02-21T11:28:47.257 INFO:teuthology.orchestra.run.trial166.stdout:+time-he.den.cod 132.163.96.5 2 u 16 64 377 40.258 -5.848 5.174 2026-02-21T11:28:47.257 INFO:teuthology.orchestra.run.trial166.stdout:+44.190.5.123 17.253.16.253 2 u 12 64 377 64.560 -1.324 0.461 2026-02-21T11:28:47.257 INFO:teuthology.orchestra.run.trial166.stdout:-eterna.binary.n 216.239.35.8 2 u 26 64 377 31.280 -0.319 1.148 2026-02-21T11:28:47.257 INFO:teuthology.orchestra.run.trial166.stdout:*chi3.us.ntp.li 204.9.54.119 2 u 32 64 377 19.032 +0.903 11.901 2026-02-21T11:28:47.258 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T11:28:47.298 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T11:28:47.299 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T11:28:47.335 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T11:28:47.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T11:28:47.362 INFO:teuthology.task.internal:Duration was 1089.489547 seconds 2026-02-21T11:28:47.363 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T11:28:47.377 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T11:28:47.378 DEBUG:teuthology.orchestra.run.trial166:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T11:28:47.403 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T11:28:47.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial166.front.sepia.ceph.com 2026-02-21T11:28:47.404 DEBUG:teuthology.orchestra.run.trial166:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T11:28:47.451 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T11:28:47.451 DEBUG:teuthology.orchestra.run.trial166:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T11:28:47.708 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T11:28:47.709 DEBUG:teuthology.orchestra.run.trial166:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T11:28:47.717 INFO:teuthology.orchestra.run.trial166.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T11:28:47.718 INFO:teuthology.orchestra.run.trial166.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T11:28:47.718 INFO:teuthology.orchestra.run.trial166.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T11:28:47.718 INFO:teuthology.orchestra.run.trial166.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T11:28:47.719 INFO:teuthology.orchestra.run.trial166.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T11:28:47.789 INFO:teuthology.orchestra.run.trial166.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T11:28:47.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T11:28:47.804 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T11:28:47.804 DEBUG:teuthology.orchestra.run.trial166:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T11:28:47.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T11:28:47.853 DEBUG:teuthology.orchestra.run.trial166:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:28:47.889 INFO:teuthology.orchestra.run.trial166.stdout:kernel.core_pattern = core 2026-02-21T11:28:47.899 DEBUG:teuthology.orchestra.run.trial166:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T11:28:47.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T11:28:47.944 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T11:28:47.964 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T11:28:47.967 DEBUG:teuthology.misc:Transferring archived files from trial166:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-21_10:47:12-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/62482/remote/trial166 2026-02-21T11:28:47.969 DEBUG:teuthology.orchestra.run.trial166:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T11:28:48.018 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T11:28:48.018 DEBUG:teuthology.orchestra.run.trial166:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T11:28:48.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T11:28:48.064 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T11:28:48.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T11:28:48.077 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T11:28:48.077 DEBUG:teuthology.orchestra.run.trial166:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T11:28:48.097 INFO:teuthology.orchestra.run.trial166.stdout: 1179687 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 11:28 /home/ubuntu/cephtest 2026-02-21T11:28:48.098 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T11:28:48.110 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T11:28:48.134 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} duration: 1089.4895467758179 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-21T11:28:48.135 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T11:28:48.223 INFO:teuthology.run:pass